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ABSTRACT 



The Bicentennial eonference on Mathemattcal Programming, held in Gaithersburg on 
November 29-December 1. 1976. examined the relationship between mathematical 
programming and the , computer. The more than 50 papers and panel discusiions 
exhibited this theme in terms of the design for3ise of. implementation of, and'impUca-" 
tiops for mathematical programming software and computations. Particular emphasis 
was placed on bringing out computer-oriented'subject matter not ordinarily presented 
•in a mathematical programming context. These resulting PrtSceedings document this 
Conference, which was jointly sponsored by SIGMAP of the ACM arid by the Applied 
Mathematics Division of the Institute for Basic Standards of NBS: " 



Keywords: Mathematical programming; linear pro^miiing; nonlinear programming- 
computer software; management-^ science;, operations research; computer , science! 
algonthro evaluation; mathematical programming educajion; ^soawiare develoDraenf' 
datJirfjases V ' 
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- \ PREFACE . 



i First I wish- to thank the Bicentennial Confpreiice Committee and our host The 
• National Bureau of Standards, for their efforts in making this conference so' very 

successful. These proceedings record the presentations, but it is not possible to include 
-the many interesting discussions that took pla^mong the leading researchers as well 

as newcomers. . ^ 

, Special notice should be taken of subjects not previously considered in confer- 
. ences on mathematical programming. In particular, tw<? subjects are: (1) interfaces 
with. the 'computing environment (hardware and software), and (2) database manag^- 
ment (mcludini matrix generati,on/report writing). Bill White is to" be congratulated in ' 
developing a program that really spans the mainstreams of mathematical programming 
Further, he did so by inviting, the leaders in these subjects and helping {hem to form 
well organized sessionsi^ > • 

Finally, and by no «means least, I extend special thanks to»our plenary speakers 
George. Dantzig and William^ Orchard-Hays. It is certainly appropriate for these 
founders of the field of Mathematical Programming Systems (MPS) to have provided 
the foundation Jor the rest of the program. - 



Harvey J.-Greenberg 



/ 

/ 



■ ' The Bicentennial Conference on Mathematical Programming was held on November 29 ^ 
December 1, 1976, at the National. Bureau of Standards. Gaithersburg. Maryland. J^^^^^^^f ^''^^^^ 
was jointly sponsored by. the Special Interest Group on Mathematical Programming (SIGMAP) of the 
Association of Computing Machinery, and by the Applied\Mathemati<^ Division of the Institute for 
Basic Standards, National Bureau of Standards, U. S. Department of Commerce. j 

. The basic theme of the Conference was the relationship be tweeiyAathematical programming "and 
the computer, and the presented papers and panel discussions exhj^ted this theme in terms of the. 
design' for use of, implementation of, and implications for mathematical, programming software and 
computations. Particular emphasis was placed W bringing out computer-oriented subject matter, not 
ordinarily presented in a mathematical programming context, Both contributed and incited papeft 
were presented, with the con^buted papers being refereed: of 54 abstracts received, 47 full papers 
were submitted, and 30 were ^ selected for presentation by 'the Session Chairmen, together with the 
Program Chairman, after the refereeing process. 

:. The Session Chairm^ functioned as an extended program committee, and,4n consultation) with 
the Program Chairman, reaUy assembled their respective sessions. Special, thanks are due them for 

•their contribution to the technical content of the Conference. And thanks go to the referees: G. 
Bennington, J. P. Blondeau, L. Bodin, B. Buzby,lr. Cooper, J. Cord, R. Cottle, R. Davis^R. Dembo, 
W Drews, J. Dyer, J. Eddington,. F. Fiala,, S. Fromovitz, D. Gay, M. Guttermaj^ Harnson, G. 
Hefley, D. Himmelblau, H. Hoc, R'. Jeroslow, D. Klingman, T. Knowles, G. Kochen^erger, J- Kowalik, 
C Krabek, M. Lenard, R. Marsten, C. MtCallum, R. Meyer, M. Minkoff, J .vlulvey, R O NeiU, T., 
mb^aka;, L. Pyle. A. Ravindran, H. Salkin, L. Schrage, M. Srm^JL. Spielberg, R. Stark, A. Waren, 
C- White, and J. Whiton. ' * 

• The Conference Committee^ did aS outstanding job. Under, the expert guidance of Harvey 
*Greenberg the administrative chores were accomplished so as-to permit the technical .program to be 
developed with a minimum of impact- from secondary sources. la particul^, Charles^ Mylander a^d 
Larry Haverly (anSri^aker ego, Joycq Draper) did yeoman's work on their Respective duties. The 
advice from Saul Gass wS'especiaily' welcome, particinarly when the Conference was formative 
stages. . ' \ - ' „ • c 

The National Bureau of StaiTdards. was a most appr6priate .location for this Conference having 
been a center of activity Jn computation and optimization- for over a quarter a centun^. And, the 
warm welcome given by B. H. Colvih,. Chief, of the Applied Matheiiiatics Division, and by A M 
McCoubrey, Director of theUnstitute for ^ic Standards, .was reflected as well, m ^^e excellent 
conference 'faqi^ties and logi^stics provided by NBS. Thanks and appreciation go to NBS, and 
especially to BiUHaU, the NBS'representative, and to Sarah Torrence, of the NBb stall. 
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REMARKS ON THE - O'CO^^OJS'-OF THEa 



-BICENTENNIAL CONFERENCE ■ ON MATHEMATICAL PROGiff^ING 
THE_,EARLy' ROLE.OF N^B.S". 



George B. Dantzig 
Stanford University 



It's a g:reat pleasucre to be here today. 
I''m ve£y glad thaf Dr. McCoubrey, '■ the 
Director, of the Institute' • for Basic. 
.Standards of the' National Bureau of 
Standcords, in his introductory remarks , ' 
told us for whom NBS worked/ namely the^ 
consiamer, . industry, the scientific 
community^ '»and educational institutions. 
It. is nice to learn that some paurts 6f the 
government truly work for iis. because some 

• of us 'had come to believe that • it, was the 
other way around. 

' Dr. _ McCoubrey mentioned the early days 
- of linear ^ programming and the cooperative 
role played- by the Bureau with Air Force 
. Project SCOOP. . I woul<J like to make this 
my theme today. 

As " some of you may recall, I was., the 
mathematical advisor to the Air Force 
Comptroller at the time that linear 
pifogranmiing was born . Thinking back .to the' ■ 
. . eaorly days, I 'have discovered a" remcirkable' 
^coincidence that I would like to pass on to 
.jfou between the date chosen for this 
..commemorative conference, November 29, 
• ■- . • 1976^ and the date ' when linear programming 
, . began. Pteliminairy flirtations with; the 
\ ideS" started in the- fall of 1946. By 

November 29, 1946, exactly 30 years, ago,"' 
o<p linear programming was conceived. • 

•During the . war I took pcirt in the 
planning activities of the Air Force.' In 
the immediate;*- jipstwar period, I ' was in the ' 
; tliroes of tryih-g to decfde whether to stay 
.. -yxth gpvefnment, begin an academic career, 

• or db. = ^research for industry. I couldn't 

• make-up ^nind. There were some people in 



Edited from a transcription taken at : the 
Conference. ' -For additional background 
information and more detailed referenbes on, 
SCOOP and early 'mathematical programming 
influences and activities, ^see Chapter 2 of 
G,*-B* Dantzig, LINEAR -PROGRAMMING AND 
EXTENSIONS, " Princeton University Press, 
. 1963. . 
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the Air Force (partienlarly Dal Hitchcock 
' -and Marshall ' Wood) who were Very keven- on 
having me stay. As bait they suggested 
that I try to the mechanize of the planning 
process. This challenge intrigued me*' In * 
the fall of ' M6, I ■ toyed with. '^different 
approaches. By November 29, 1946^ the idea 
came- that perhaps ' the Input-Output 
technique of ' Leontief . (for which he 
■received the NoiDel Prize/* in 1973) could He 
suitaibly generalized. In the-, winter of 
''46, my work began in earnest; by June of 
■47, the linesor programming model as we 
know it today was well along. 

bur research from early 1947 on was' 
influenceii by a conference arrcinged by 
Aiken at ..Harvard. It was here that" 
Mctrshall Wood and. I became first exposed to 
the idea of an electronic computer. It was-* 
*a wonderful conference.. Although it was 
. only a gleam" in the eyes of the speakers^ 
they talked about ' electronic computers as 
if they • really existed 'and indeed with 
capabilities very much as- they have, today.." 
I was overwhelmed with the potential of 
this new tool. To appreciajte what happened 
in the- early stages of lineax programming-, 
it's well to V remember that we beli^eved that 
computers would become practical within a 
year or two. We acted accordingly. . ' ^ 

... - . ' ^ * 

"It would be interesting to speculatie how - 
many importatnt developments might -never 
have happened if one kne^ that it would 
take "almost two decade.s for powerful 
.computers to become a practical r€fality. 

From the beginning, itf.. .was' this gleam in 
the ^ye of the designers. ' that fast,^ 
practical computers. really would soon 
become available that * motiva.ted 

computational development of linear 
programming. It r-esulted in the Air Force 
decision .to mechauiize the plamning 
process. The name for the effort was Ai5; ; 
Force Project SCOOP, standing for 
Scientific Computation of Optimum Programs^ 
The Office of' Naval .Research, ^aorticulaurly 
Dr. ^ina Heese, who is well, known to mciny 
of you, played an "important * role. . She 
introduced ■ us to Jb&n. . Curtiss .and . his 
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piathematics group . a^:"^ the Bureau. * Her 
office sjubsid^zed, related research, 

; . Oiir ":Hgr;oup was not- technically equipped 
to supei^i^xse' or evaluate- the^'development of 

^ computers.? We turned to.'-^.he Bureau, of. 

. Stiandardsjto serve as oiir ' technical agents^ r 
, ^ to? ' kee^r; '■: us informed cibout • computer 

, *'deve'lopmen^is . And so it' came about that ^ 
became one of 'the behind-the-scenes 

> sponsors of the eadrly ' development . of 
computers. The Air Force Comptroller 
transferred huge sums of money, to * the NBS 
for this jfjurpose. There ^ were close 
contacts with Sam Alexander ^ whose .group at 
NBS built the SEAC- Air Force money helped 
NBS fCmd .the" building^^of BINAC, UNIVAC, and 
also some IBM comppnent research* I don't 
claim that SCOOP was the only spofisoir 
, (directly or indirectly} in this, field; 
there were others, for example the Bureau 
of the Census; nor did SCOOP sponsor the 
building of .SWAC under "^cirry Huskey at 'the-, 

, ^Bureau's •Institut;e of Numerical Analysis at" 
UCLA.. 

•I am particularly grateful for the 
advice of Albert Cahn who worked for 
Curtiss. He - recommended tVo key people 
. whom he said I should consult reg2Lrding the 
'relation of linear .. programming* to 
^ economics, * mathematics, and nmerical 
analysis: the* 'first was the economist, 
T jailing Koopmans; the other, -the/ famous 
mathematicietn, Johnny Von Neumemn. 'in June 
of 1947, I went to the Cowles Foundation iat 
the University of Chicago to-* see Koopmans. 
This contact initiated his Iriterest and 
soon the interest o-f other young economists 
(many now well known) in the relationship 
between mathematical programming ^ and 
economics. In ISVS'Koopmaois received the 
Nobel Prize for his contributions to the 
theory of resburce allocation. In the fall 
of 1947, I went with Curtiss; to -Princeton 
to see Von " Neumsmn at th^Jnstitute' of 
Advanced' S.tudy. In the course of our 
discusssions. Von .Neumann stated the 
duality theorem, related- it. to g'ame theory, 
and made other pbse'rvations that, laid the 
mathematical foundations. -(The _ history 'of 
the duality concept makes^an "Vitteresting 
♦ stoj^ in itself ^ ^ 

In.J-une of 19"487 'John Curtiss again 
introduced \ me, this v time . to his 
brothear-in-law, Al Tucker ' at Princeton. 
-Soon, thereafter Tucker with his students 
Harold Kuhn and \David • Gale started a 
. . seminetr .which resulted in their^ well known^ 
contributions . to duality theory, game jj 
theory auid nonlineaur ' progr conming * Von ( 
Neuman and Tucker spearheaded the' interest ' 
of mathematicians. ^ ' " 

s While, tbis academic interest was 
growing^ work b^aui in eacmest within the 
Air Force to * mechanize the planning ' 
process. During the period » 1947-52 there 
were 'two main branches to our efforts ~ 
one prac-fcical, the other theoretical. Jhe 
goal of . the practical branch was to 



impn^Bment quickly 'and to solve rou;t'inely 
very^ very large dyr&mic programs required 

- f or* plamnihg • These.* (systems were^sb large 
that, even by today 's .standards, they would^ 
'^.ie beyond^ anyone * s capability to optimize 
as a. single linear program.. For practical 
plannin^g* Marshall Wood and I invented a . 
hierarchical stepwise optimization scheme 
called a "triangular model" . -^We echelonned 

. the activities of* the' Air Force in 
"paras it i"c" order ~ namely, on the top rung 
were the combat units .(which took from' 
everybody - but . gave^ nothing in* return) ;. next 
be^ow them -were<-the support uJrits irf the 
combat zone <which took from everybody ' 
below them but gav^ nothing, in return) ; and , 
so forth ' down to support -units that 

i^rj^cruited and ^trained personnel aind the'- 
units which ^"bought • and distributed 
supplies.. • Although called * a triangular 
model it could" be applied to general 
matrices Ipy first m^akmg a triangular 

.approximation ancL then making iterative 

fCorjrecti^ns by adding on . additional lower 
rungs in the hierarchical ordering, * Mvurray • 
'-Geisler's^ group formulated Air Force 
plauining 'problems in this manner. Sted 

'Nobel emd others extended the formulation' 
to sectors of the national economy, ^ 

The triangular model . scheme wa^/ 
implemented, computationally in the spring 
of 1949 by Mike Montalbano of the National 
Bureau of 'Standaords. On, the • plane coming 
to this conference I ran into Mike. 'We 
reminisced about his achievement. Mike did 
a veary remarkable thing: he used IBM 
punched card equipment, the only ecfUipment 
available, in a way that .no one had ever 
used it before* tfo -iterativfely carry out* 
the programmed steps, he processed the 
cards' through a sequence of machines: 
tabulators, .sorters, '^reproducers, 

collaters, and IBM 604 's, all atrranged'in^a 
great big circle. Because the equ^t^efit 
was unreliable, he also processed the c^d3 
through two pieces of similar -equiprnferitg 
serially in order \o have one machine checV 
thW computations of the other. 

One day during the development of the 
. computational system by Montalbano, «^?assily ' 
Leontief visited Washington. I i asked. 
Wassily if he would like to come oh^over to 
the Bureau of Standards to t2i}^'"a look 
(they had a - downtown office where tests 
were being run)'. .'It was a Sunday, and Mike^ 
was pleased to^put on a demonstration of 
just how wonderful his system was. He 
pointed out that- the equipment was 
unreliable cuid how he had to get after IBM 
regularly to make it work. He pointed out^ 
the features of his ' system for checking 
errors. "For example", he said, "See -those 
tV70 reproducers, one following the other? 
If the holes r pxinched by the second do not 
match exactly those^punched by ,the .first, 
the second reproducer will stop, turn on 
red lights, and we will know that the last 
'card processed is wrong." 

As the csurds were running through the 
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second reprpducer,' Mtke said,. ^ "You see, at 
-this ^int eveir^rthiYig * ds'^'. working^ » fine y 
^t^ere-* are' jio red lights.*",- * 

But on t"he side to me^ he said, "Isn't it 
making a loud *cruncJ;aii'g''no-ise?" - ' 

As he pulled the cards from -tlie output 
hopper,, we saw that the>. columns were laced 
full of holies — ' indeed 'they were nothing 
but confetti. The cards » were: 'clearly 
ovferpunched, but no red ligh-ts had turned 
on.. Leontief tapped me on the shoulder and 
said, with^ his Russian accent, /There, 
there ,r I understand these things. . Don't ' 
v'or2ry\ A machine, is like a voia'cin, very . 
temperamental," This stqry is a classic, 
example • of System Antics * ; — tte science of*; 
•.why complex systems work poorly if at all / 
Something unforeseen always • happens. In 
this cas^ . someone the night' before had 
ch^ged . the stcindard positions *^ of the 
control brushes on. the reproducer arid,j, 
forgot to change them back. 7^, 

, • -it J* 

Turning now to the^ theoretical brcinch, 

namely research on tlie linear programming 

iaodel, here also the Air Force turned to 

the Bureau for help. Jack Ladefmcin and his 

group at' the Mathematical Tables Project in 

New .York, using hand calculators, solved' 

Stigler's nutrition problem using the 

Simplex Method (December 1948), This- was 

the first real test of the method. To do* 

the computations, Laderman handed out small 

worksheets. These completed worksheets 

were pasted together to form a huge "table 

cloth" . . In going through some old 

correspondence recently, I "found a letter 

from Oskatr Morgenstem ~ he wanted to come 

down to Washington to see the famous table 

cloth". I wonder whatever happened , to it? 



In ea^J-y 1950^ Montalbano'^with Corky 
Diehm wrQ£e-the first Simplex •Code cind made 
successful runs with it on the* SEAC 
computer. 

There wei^e* others at the Bureau well 
knowr)r''to you -who contributed in a major- way 
to the early development, Alex^Orden, who 
is;;fcere today, was with . the Bureau' for a 
biiief period before .he joined our Air Force 
group. • There^ was Ted Motzkin, Alan 
Hoffman, and *Leon Gai^nen. George Suzuki, 

'\now with the Bureau, was. with us at^ SCOOP, 
as was Joe Natrella, whose wife Mary ^."s 
still at ^the Bureau. During the eaxly 

. 1950'-s, Hoffman, -Mannos, Sokolowsky and 
Wiegman ran comparative , tests of the 
Simplex Method with alternative algorithms: 
At I.N. A.., Motzkin, .Raiffa, Thompson, and. 
Thrall . developed the Doulple Description 
Method; Mptzkin and Shoenberg developed 
the RelcLxation Method; and. C* B, - Tompkins, , 
the Projection Method,' 

Thoseu were • the days to remember! I, 
personally^, .^am grateful to the Btire^u for 
(1) ^ t he ir^- pioneering the construction of 
computers. -^and^ their sponsorship of others 
in the field; (2) their design, of the -first 
linear 'programming software; (3) . their 
contacts with economists and mathematicians 
which br9ught about eairli^ interest by the 
academic * community; . (4) their sponsorship 
of symposia on mathematical programming; 
(5J* their research on mathematical 
programming theory, algorithms, ajid their, 
making comparative tests.. These . 

contributions played a major role in the 
early 'rapid development of the mathematical 
programming field, 

\ 
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EliEHGy >!DDELS AND lABGE-SCALE SYStTEMS OPTIMIZATION 

George B. Dantz£g and Shaileudra C. P^ikh 
V Stanford University \ 



The optimization of . large-scaJLe dynamic 
systems represents a centriaJ. area of reseiarch 
"whose successful outcome could make important con- 
tributions to the' analysis of crucial national and. 
vorld problems.. ^Although a great number ' of papers 
have-been published on the theory of Solving large- 
scale systems, not much in software exists that ^ 
can successfully solve such systems • - We believe 

• that there has been little progress, because jthere 
has. been little in the vay of extensive ex5)ertaen- 
tation comparing methods under^ laboratory-like con- 

. ditions on representative models. At Stanford's 
Systems Optimization Laboratory (SOL), to bridge* 
this'gap between theory and application, ve 

(l) develop experimental software for solving. 

.large-scale dynamic systems, 
f2) systematically con^are proposed techniques ' on 

representative models, ^ 
(3) record and disseminate information regaj^iing 

exper imeSital results. 



PILOT Energy-BcQDomlc Model . 

Dynamic models that describe the inter- 
actions between the energy seclior and the general ^ 
economy help in providing a focus to our i*e search 
in experimenting with J^ge- scale optimization . 
models, ^ Models "of this type are under development 
by a number "of groups to^study the energy crisis 
and a probable, future crisis in the area of food 
(agriculture)- Developers of these models could 
mgJte effective use techniques for solving 
large-scale systems, if they* were available." 

Today* s policy makers at industrial, 
governmental and international levels are* faced 
with the decisions on providing the. needed energy 
in the years to come at acceptable social cost. 
Such decisions must take into account many com- 
plex "i^nteractions related to the technology of • 

- w 
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energy supply, environmeuMl side effects, energy 
resource cpnservation, * etcV^, ^as^well as the natiotial 
'. welfare considerations of ^ uheiqployment, inflation % 
and living standaMs, * • ' ' 

Some of th^ ' importfiLnt qi:ysstions that must 
be considered in detail in the formulation ofj^he : 
energy. policy (or policies) are the fo^llowingi 

.(1) Are wja using up our cheap energy resources too 
(quickly? -. 

(2) Are .we maiing soiff icient investments now so ^ 
that liew -energy technologies will come Into 
commercial operation when needed ih' future 

• ' years?" / \ » 

(5) Do we have sufficient physical capacity tc5 

build thi'required new plant and equipment in 
the energy, and non^ergy sectors without - 
seriously hampering growth in consumer con- 
' . suir^tion, or will some sacrifice in consump- 
tion be necessajTy? \ - 

(U) What are the variou^ energy .options under 

different patterns of crude oil import price 
realizations?" 

(5) What will the short and long term impact be 
if bil and gas discoveries are less than pre- 
dicted? : ' 

(6) Can we find an energy policy that is robust, 
i.e., one which hedges aga i nst variotis con- 
tingencies? 

It is our belief that dynamic mathematical pro- 
gramining models can, .at the veiy least, provide 
analysis and information on these and other ques- 
tions and can -substantially improve the iinder stand- 
ing of the interact io^ that must be considered. 
Such models have been developed at HASA, at the * 
Electricite de France and by various groups in the 
United States. . • . 

In the Systems Optimization Laboratory at 
Stanford we have under development a linear pro- 
gramming model, for assessing energy-economic op- 
tions in the United States, called PILOT [7]- It- 
spans a wide spectrum of activities of the economy, 
from exploration and extraction 'of raw energy to 
' industrial production and consumer demands for all 
'"goods and services* The data requirements there-- 
fore cut across many different sources— consumer 
surveys, import/export and trade balance data, 
manufacturers surveys, juining data, input /output 
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and cap^al *coef f icients^, energy consumption etnd 
substitution data, energy technology ^^ta. from 
Brookhaven National Laboratory and oil and gas 
exploration and production dkta. Hence^ there is 
a nbntri vial- problem of achieving consistency and." 
of s.electing a meaningfur leve-1 of detail so' that. 
*the model stands ag* a whole rather than as a con- * 
glomeration of parts that could; collapse under 
^3jreful Analysis.*- We ■-believe that the initial ver- 
sion of the- model being built wil^ meet this ^ test. 




sf 

t) ] .The PILOT model is a statement in. physical 
sf low .terms,, to the extent poss^^i^Clv of the>.b3:oed ' _ 
technological interactions w>thirr-^d^a.cross*the '^'^'^ 
Rectors of the economy, including,^ but in ; greater , 
tail, the energy se;:tor. ^a' tlypical run of the * 
. h describes what the country could achieve* in 
. physical terms over the long term, say' years. 

• ■ . • ■ * 

. STv pre jljimi nary version of the model has' 
been -completed and s"everal useful scenarios have 
been runV^6}». In 1977, improved. -^^ejrs ions of the 
model wi^Il be developed, w'ith more .de"Kail regaxd-' 
ing exploration and. extraction by regions, more 
detailed modeling of Various co'nver^i^n processes, 
better reoresentation "of fo re ign.« trade, substitu-- ' 
tibn, financial 'flows and- the effect pS* prices on^ 

• dem^yai^and proSiiction. . ' ^ • ^ 

The initial" version of the PILOT modisl :is ^ 
an eight cjeriod,'' hO year model vhichjias' approxii- : 
mately 800 .constraints and 2000' vari able s, ' ^"^ 

^ -The model is 'a description in input/output .* 
terms of the industrial processes of the economy 
.^and 'the demands for consumption, capacity forma-, 
tion, government ^services and net exports'. The'?":?^*^- 
description of the "processes that provide usef£l 'v. v 
energy to the economy ^constitutes the detailed 
^energy submodel. This consists of technological 
"input /output descriptions of the raw energy ex- 
traction and the energy conversion processes as •. . 
well as the energy import and export activities. ' b 
Four linkages interconnect the enet^gy sector and 
the rest of the economy: energy demands of the. 
economy, bill of goods needed for energy process- ^ 
-ing- and capacity expansion, total -manpower avail- ' 
able to all sectors ^ including energy) and a trade ^ 
balance constraint which requires the equating of ■ ^ 
total exports to total imports when these items 
are evaluated in 19^7 dollars over successive 
five year periods.. See Figure 1, which shows.. the 
major blocks cf coef f icients • in a time period, . ^ 
and » its link to the next time period show below . 
and to the lef t ^of the dotted. *lines. 

. As noted" earlier, the" ^uations of the ■ 
model' express the ba-i.ances of various physical 
flows* For the energy sector, the balances of ■ 
coal, oil, gas, etc. are each expressed in BTU 
units. For the economy, the units are 1967 ''^ 
dollars, ^^-hich are obtained by weighting the 
underlying *phy sic al flows of goods and services . 
(assumed to' be in fixed prooortions) by 1^6j prices 
per unit, *^ B . 

The industrial sectors .of 'the economy are- 
represented by a 25-<5rder input /output matrix. The 
sectors sire grouped as .follows: ♦ 5 energy sectors, 
1* agriculture, 1 nonenergy minj^ng, . 5 energy inten- 
sive maniif^acturing, U energy nonintensive manufac- 
turing, h services and 3 capital formation. 



Consumption is modeled in. -terms ^ of the con-, 
sumption vector of ^the average consumer. This *. ' 
sector does not Aave «a fixed bill -of goods per , 
capita but varies as a fui^iation of a. parameter ' % 
. J! representing the "real- consumption - income a^ttaineS 

, per capita.. 'Base'd on ^n analysis of historical* 
. - ■ data, consumption of "aHy^tem. as -function ojf aver- 
' age consumption income isVnearly linear*!^. 

■ Capacities- .for each 'of the l3 nonenergy • i.. 
sectors and, all of the\ energy processes are differ-" 
\ entiate^ f^om ohe-'anotiier. The- heterogeneous cap- 

- ^t_^i %quip*]ae5tt of'tne nonen^e^gy sectors is ^depre- 
c i at ed,^^he r eas. the ene rgy ftc i lity « cap ac it ie s are - . 
assumed insl^ead to have undepreciated, fixed life- 
times. 'Coni?feuctj.on la^ are used- to specific he 
time ilj- takes to /build new- capacity. These con- l 
struction lags may' be chosen individually for all 

, 18 nonenergy sect'ors as veil as for all energy • ^ 
facilities. "j^- ^ ^' 5j 

/ The. exports are treated as final der^and 
'\te!;[is. The imports are** considered in two parts, 
• competitive^ and noncompetitive. The nozjcompetitive 
•impo3rts are' for those goods ,^d services for -A-hich - 
no domestic substitutes exist. 'They are treated ... - 
as a part oT the technology of the consuming in- 
dustrial sector. On^he other hand, competitive 
^ impo?t^of goods and servicjss for which' domestic * ' • 
substitutes do. exist are 'treated as activities that 
can augment the domestic production. Finally, «.a . 
trade balance constraint ties together the amjncfts ' 
of all imports ismd exports. Typically^ . we hav^ r 
"assumed over a five year ''period that .the value of 
total .exDorts be matched by that of imoorts. 

The labor force is assiinea. to be^^.exogenously 
^. given. Also, average labor productivity is assumed 

to grow at sm exogenous ly - given rate. In sample 
,' runs, the "-standard of living" attained appears to.', 
be very sensitive to this ^factor. In the base 
case, 2/^ per year .productivit^y. groy'th is .assumed, "• - 

■*'The detailed energy , sector contains the; con-^ 
ventlonal energy technologies,.*, such as oil /refinery, 
coal fired, plant, etc.," as/^-ell as new technclogies, 

- -such- as coal synthetics"^ oil shale, plutoniux. fired 
. reactors, etc. ' .. • . 

^-1^^' . ■ ■ ■ ■. 

. 5he description of the energjr sector^ includes 
an accounting of ^reserves femsiiiiii^ of three ex- . 
haustible energy resources> oil, gas and uranium. 
For o;ii and gas," finding-rate Tunptions' are used 
to specify the amount of oil in place -aija gas re- 
serves to.he^found for a given amount of drilling 
''-effort. . The"^ level of drilling effort is endoge- 
nously determined, - The advanced (and expansive) 
"Siechniques of secondary SLnd tertiary recovery are 
also defined in the model. Alaskaji oil production 
•and .the Trans -Alasketn Pipeline System (TAPS) con-.* 
struction are assumed to.be exogenously giyen." For 
natural uranium, increasing facilities and 'man- ■ 
pO'vieT are re^quired to extract a ton of ore as mor"e 
and more is extracted. In- particular, progress- 
ively higher amounts of uranium mining sind tailling^ 
capacity are needed to - process the poorer grade 
/or^^per pound of uranium oxide obtained. "While, in 
principle, generalized linear programming could be "• 
used to model the nonlinear functions, > we, in^faqt, 
replaced -the nonlinear functions by broken line 
fits. .. - " o " 
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• ■ ^ The PILOT model 'can be used in' conjunction 

• with" jiny .^esired sp^a^ objective. ^Consideration^ 
bf,;rep.s enable'- alt eiriitive ob^cJesJ^tiyes^a^equires , . . 
l\Lrther investigation Some objec^ves -tq^ re-. . 

♦quire l^eir. eKMe ssi on partly, ip the form' of extra 

constraXffts as^^ll as in the values of the «<:oef 
.ficients' of t&e maximand. The objective 'chosen 
in^the be^e case in the 'initial version of the ' * v 
model, is the undiscounte(f"suni of the gross* na- ^ 
tional consumption over all hX) years, "subject to, 
(l) a "monotonic per capita'^ constraint which * ^ - 
states that the average per capita consumption igust 
be no nde creasing ;;over" time, and (2) an initial con- 
"dition which -sets a. lower limit on .first- pe;?iod 
consumption, Eicperimentation with oth^ -maximands 
is possible. For example, we have exp^imeated * 
with discounted gross naJtionaX *consximptit)n. 

The objective • of ' PILOT is designed to per- 
mit one to determine feasible solutiodS -to our 
economy— in particular, what level of investment , 
(in physical terms) both in the energy, and non- 
energy sectors is necessary in order to have as 
high a standard of living as possible for the grow- 
ing population. 

'•>?>v ■ . " ' - \ ~* 

Once the physical flows are determined, it 
is possible to solve a related financial* invest- 
ment model. The financial flow model calciilates 
a system of prices, taxes, salaries, profits, in- 
terest rates, etc. that is internally consistent . ^ 
in the sense *that all economic agents — consumers, 
producers, government, etc. , --receive sufficient * 
'monies to pay their expenses for the specified' 
physical flows. The prices generated by the' model 
can be adjus?ted to be, at the same time, nonin- <' 
^flationary, jiiLe., to have the same buying power as 
base year prices; We also are giving some 'thought 
to. incorporating in the model production and demand 
functions to adjust input/output coe^icients as a 
function of prices. In the initial version of the, 
model these coefficients are fixed^ however. 

To illustrate some of the output of the 
modely a typical base case assumes a 2ff> growth in 
labor "productivity, 20^ limit on the total amount 
of energy * purchased overseas, certain*, limits on 
the rate of growth of coal production, etc^ A base 
case run computes consumption iJ^icome (income after 
taxes- and 'savings) . In 1975 this income per capita 
(in 1975 dollars) was. about $^500. ■ Based on these 
assumptions, the tnodel states it is possible fqr 
the country to havev,^^uture consumption- income 
per. capita relative to^ consumption per capita' ,in 
1975 as follows: ^ 

. 1975 1980 1985 ' . 1990 1995 2000 200 5 2010 

• . l!o . 1.0 .1,2 1.6" .1^7 1.8 2.0 '2.2 

This •possible future can be compared with that ob- 
tained from J another scenario, which is the same 'as 
the ba^e case but reetrficts the use of nuclear 
power plants. This naturally results in a lower 
achievable per' capita consumption income. Relative . 
.to 1975, the results are as follows: , ■ 

, 1975 - .i980 1985 1990 1995 2006 2005 ' . 2010 

1.0 ■ 1.0" 1.2- 1.5' 1^5 1.3^-^1'5/ 1.5" > 

Comparing ^he two scenari6^s at'^'^ihe Jtear 2010, we / 
have 2.2 vs. 1..5; i.e.'^ tne "nucleaari restriction -• J 



,coul"^ reduce the "standard of living" achievafble\ 
by '2010 ^by 5056* ' 'This conclusion has been criti-^ 
cized 'because the model ^sumes.that. consuiltotiorx~. 
'pOctterns 'ot people at difJerent incotae levels wlir 
remain "unchaiiged (i.e. /I^e^; -won^^t pipeCctic;^ conser- 
vation or change the'ir lirfe .siylepf-" and -"that pro-r 
duct ion methods will be no'mcjre efficient' in the 
use .of energy in the future than -'they- a*S;^.todp.y. 
"This- ^criticism we feel has merit and we* fiie;^ ac- 
cordingly^ considering revisiotis in the mpdel^, to" 
include more substitution aid conservat5jbn-rpossi-*» 
bilities. ♦ " ' ■ ^ * ' 

Because of excellent liaison with- other , 
groups workiner. in the -energy field, we. anticipate 
that the propo^d physical flow mjrf^el will.contri- 
.bute to the formulation and solution the more', 
detailed specialized models^ under development 
else-fcrfiere. In particular, the PILOT' model is one 
being compared with other models by^t^e newly 
formed U.'S. Energy J^deling Fortim .in-'its- examina- ■ 
tion of the feedbacks effects from the energy sec- 
tor on the economiCjgrowth. v -. •• 

Solving Mult i -Time-Period Models' ^- 

Solving energy models, by commercial linear 
programming software is ^provingl to be expensive. 
Large-scale techniques, such as those under devel- 
opment ,at the Systems Optimization. Laboratory, 'are 
currently under test to see if they. can help solve 
these models more efficiently." '* ' . 

Conceptually, the decomposil^ion principle ^ . 
[-8] .has proved to be a natural- approach to breath- 
ing^ up iarge systems and to decentralized, deqj.'feion' 
taaking. ;.So far, computational ^perience has tee^ ^ 
limi'ted, 'but it is known that several devices can. • 
be effectively combined^ with the decomposition 
principle to "accelerate" the iterative process.' 
Classic reseaJTcla along these lix^es can be found in 
the work of Rosen [17]> Beale^'[3]/ Gass (9], Bell 

Abadie (1}, Bennett [-5],' as well as in the * 
joint work- oif Wolfe and* DaxLtzig [8]. Areas of 
SOL research include (l) intertemporal models 
with staircase' s true fiires, . (\2^t he continuous 

* simplex method for linear ccmtol problems^ith 
state-space -constraints, and;:^1(3) general large- 
scale dynamic nonlinear problems.- 

Recently, experiment's have been conducted 
at SOL comparing, the Decomposition Principle ap- 
proach with a special .variant* of the simplex method 
known as Generalized GUBl'l; These tests were limited 
in nature but indie at ed/'-;that' Generalized GUB is 
superior. Our research" on .dynamic systems is 
therefore ■ examining vaxiants of the .^.implex method 
as* well as special methods for decoupling stair- 
case and block-triangular 'systems. See Figure 2. 
We plan* to compare, these, approaches with the nested 
decomposition, algo'rithm of James . Hp and Alan S. 
Manne for staircas'i*' systeins [12]. ' * - 

Staircase systems have historically proven 
to be very difficult > .usually requiring a dispro- • 
portionately . large , number of simplex iterations 
to solve. ■ - • 

* ■ ' ^-^ 

This section is-;based on a summary prepared by 
J» A. Tomlin. .r . >' ■ 




FIGfURE 2. The Staircase Structure of the PILOT Energy Model 



J* A* Tomlin of SOL has experimented with 
a partial decoupling of time periods vithin, a 
model by relaxing the intertemporal constraints^- 
The expectation is that such a relaxation "Cril^ 
re^^t in a more^ easily solvable mo'Sel, "whose 
solution* can be -used as a starting point fpr the 
real model, using some. "gradual" ^pproadi to re- 
store the intertemporal ^constraints. As might be 
•e3cpected^ the results .pf this approach are quite 
probleTn depejidient^ and sensitive* to the degree and 
the kind of relaxation employed. It appears that 
tightiy constrained economic planning models^ oif 
the type available to us for these experiments, ■ 
require . a more sophisticated approach, • ' Other - 

. typtes of staircase models are being acquired to 

" further test this. idea. 



One of th€t'more promising methods that - 
have been investigated for reducing solution time 
for dynamic models involves several modifications 
to -the simplex method desi'gned to take advantage 
of the special properties and behavior of such 
models. The essential property of interest' is the 
tendency of the same' type of activity to he hasic ■ 
over -several successive tim^ periods. It there- 
fore seems desirable to introduce a profitable 
type of activity in as many time period^ as possi- 
ble simultsineousiy. To do this M.A, Saunders^ and 

• JiA. Tomlin have explored variants of the reduced- 
gradient method (a/ nonlinear programming algorithm, 
already implemented by Murtagh ^d Saunders in 
MINOS [li^]) on these 'linear problems to change 

■ several, nonb as ic variables simultaneously^ in 
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\ contra«: to- the standard simplex -method whi'tih 

\ chaag^ onl^ one no nb as ic variable at"" a time^^ To 
^ ehsur^that the correct ndnbasic variables are* . 

. used^ a "sp«ial. pricing" techniq;ae; is employed/ . 
* • *WhenJihe problem is read in, simiiar^.actiV^ities 
■'^ V in different time periods are identified (from the- 
column or varfabie n/ynes ) - an4 ' linked by a-.-Circular 
^\ list. "Thus vhen an activity is. priced out .and ^. 
* found to h%ve a favorable feradienti, the cprrespond- 
ing vectors in^other time periods can be easily 
•found and examined, and,* if satisfactory, included 

- "-as^ candidates to be ch^l^ed. It is then possible 
' to" make a step vhich introduces 'an activity in 

several successive tliae periods simultaneously, 

. ' Preliminary experiments with the above 
^ appj^ach have? led to a -reduction of 20-30^ in 
Phase U iterations vhen compared to.^the standard 
simplex method applied to the type of economic ■ ■ 
X>lann1 ng models referred to above. It is ^ear 
^hat maS^ tactical va^3^a■£ions of.i:he schene need- 
' ^o be studi'ed. „' 

If it is advantageous to bring in aa activ- 
ity simultaneously in many time periods, 4;hen, . 
conversely; it ^should be advantageous to be able 
V to also force an unprofitable' activity to itq . 
^tlower bound in several time periods simultaneously, 
■^is is rather more difficult, since one cannot -> 
tell whether a whole, group of variables can reach 
*^;heir bounds while maintaining a feasible solution 
^ ■ (at least^. not without dncurring a heavy computa- 

- tional cost) . The approach we have implemented 
identifies, groups of unprofitable nonbasic activ- 
ities close to their bounds and fo^ms a direction 
vector scaled in such a way that if one of these 
variables 'reaches its botind then all of them do. 
This method has had some .success in reducing the 
iteration count wh'en combined with the "special * 
pricin^^ described above. -Again many variations 
are possible, and very' considerable further ex- 
perimentation is required to refine the methodology 

. ajid expand on the promising results achieved so. 
far, 

"While we have concentrated on means -of- 
improving the solution' path (iteration count) 
above, another means of improving solution tech- * 
niques for staircase models is to speed up each 
step of the simplex algorithm by taking advantage 
of the special structure of J2ie basis for such 
problems. As early as 195^, cTfe, Dantzig [6] 
pointed out that such problems exhibit an -"almost'* 
square block-triangular basis structure which 
[ could' be decomposed into a product of .a true 
square block-triangular matrix and another matrix 

. with only a few column if differing from the'unit\ 
' matrix. The advantagja. of this procedure is that 
sqaiare block-triangular matrices can themselves 
be very efficiently, decomposed to ,give .a .very 
sparse factorization of the basis, A version of. . 
tliis .method, employing modern .factorization- tech- * 
niques, has been implemented at SOL by A,F, ,Perold 
(a graduate student) and is now being tested on^ 
problems of significant size. Early indications 
are that this method of^ handling the basis can in- 
deed be more efficient than a direct .treatment 
which, does not take the staircase structure of 
models into account. ^^Perold^s code is based on ^ 
SOL^s LPMl linear programming code^. as was the ' . 
nested decomposition cod.^by J.K. Ho and A,S* Manne 

, [12] for the same class of problems. This should 



facilitate com£arison .between the- specialized 
simplex aivJ decomposition approacijes «to these- 
models, " - . , - . ^ ' 

' An -alternative 'to ail -of the above numer- 
J^caJ^tree^tnents.^ of "discrete mTiiti-tiTie-period 
models is to attemp't* to solve the, underlying con- . 
tinuous time' problem, ^which'can be tiiought of as a 
^linear control problem^-^th state-space constraints 
G.B. Dantsig^^g^ R,E*r Davis are investigating a 
'^continuous simj^S^^^^a^hod" for such problems . Al- 
' though progress has been made^ much work remains 
to be ■ done . " • ' ' - 
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. THE CHALLENGE^S'^S^NALYTIC USE OF COMPUTERS 
■ FOR GLOBAL -PROBLEJIS 
\ ■ . * ' ' ^ 

VJilliam Orchard-Hays . ! ' ' 
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This year the United States is two 
centuries old, as you ma^ be tired of 

• hearing by now. Even if one goes back to 
the early English settlements, this' land 
is only about three and one-half penturies 
old. Th'at' is not very long, really. In ' 
terms of generations, about 12 to Thet . 

small city near Vienna where I. have been 
living for nearly twa years goes back to . 
■ the tenth or eleventh century. There are 
many active monasteries in Austria and 
some have been in continuous operation 
since the^ighth or ninth century. So we 
are a young nation in some ways; yet our 
political institutions are among the -ir* ■ 
oldest in their present form, ^and in many 
areas of technology and business we^ have 
led the rest of the world. 

The impact of the U.^S. on--th^ world 

_Jfcias been tremendous. One, must live for 
awhile outside North America to begin 
really to understand this. Other peoples, 
have watched us — not necessarily their 
governments — much longer than we have 
watched iihem, and still do to a greater 
degree. I7e began to be a world power- 
probably about th'e time of the Spanish- 
American war, just over three-quarters of 

century ago. After World War I, U.S. 
business became a dominant force . in many 
areas and grew with the world. ■ And it 
has grown. I was born just* before the 
end of World I'/ar I and the population of* 
the world was ^hen much less" than two* 
billion, or less than. 455 of what it is 
today* Approximately the same ratio 
holds for the U.S. and the effect is . ^' 
noticeable. 

■ During and after World War II, of 
coTirse, the U.S. became the mightiest 
/nation on earth — in almost any way you 
want to measure* except sheer population 
figures- Yet it was five years after 
that before computing as- we" know it to- 
day could be said to be in its' infancy. 
(Vuegraph ll) There were, of course, 
antecedents going back much .further, . but 
these arewnainly of historical interest. 
I started in the field at the RAND 

^ Corporation in January, 1951. It was 
three years .latter before we had a real 
computer and we wejje simong the first. I 

" had 'lived' almost half a lifetime before 



getting into computing and still my career 
^has spanned virtually its entire history, • 
So our field is very young indeed. . 
... . Nevertheless, the accomplishments of 
the computing field during its first 
quarter century are extremely impressive. 
Furthermore, with apologies to our many 
foreign friends and professional associ-' 
ates, "computing* is almost exclusively ' an 
Ameriqan development- There are three 
technologies in which we are still clearly 
supreme:, telephone systems, commercial 
aircraft, and computing. (The list is no 
doubt longer but, I fear,- getting shorter 
as time goes on.) One flies almost every- 
where- in American aircraft, and foreign 
computer manufacturers have all but given 

/Up — with the possible exception of Japan 
and the new imitative line which the 
Eastern bloc is attempting to create. 
Some very good sbftware has come out of 
France, particularly in the MP field, and a 
little from "England, but almost always in . 
connection with American, manufacturers or 
'other multi-national corporations. The *■ 
bulk of system software comes from the U.S. 
and a large* part of application software.- 
(Vuegraph 2, with extemporaneous dis-- 
cussion of...evblution of. computing 
technology.) " ' * - 

I think no one in the 1950s really 
xinderstood the amount of effort required to 

. develop systems, applicaftion packages, 
efficient compilers, and so on,. Almost .* 
nothing worked, right during the whole 
frustrating decade of .the "eOs. In retro- 
spect, it is not surprising and we could, 
all be forgiven, I think, .for a little 
self-praise tha,t so .much was actually ' 
accomplished. But those frus^trations 
often left a bad taste, a deep skepticism, 
a rigidity of method and, in some cases, 
bad feelings- One can still see the 
effects of this. 

.But the '60s are behind us," and there 
> is no point in dwelliiig on the mistakes, ■ 
failures and unrealistic estimates. The 
aolid accomplishments far. outweigh them, - 
In fact, the field has progressed so 
rapidly that it is now possible forgone or 
twp people to accomplish in a ^couple of 

'weeks what would have been considered, 

.dnly ten years ago, a' sizeable project. I ^ 



phave oompleted several su^ch tasks in the 
past several months, ^nd surprised even . 
myself. But this doesn't happen always 
and everywhere. It is surprising how 
much" computing ' is • stilly don^» in old- 
fashioi?ed ways {if we- can. use tl;iat. term 
for so young a field) and how slowly newer ■ 
techniques spread. At cne time, I fought 
operating systems — not because I didn't, 
understand the '.problems but , on the ^ 
contrary, r understood them very well and 
didn't trust anyone else "to solve them in 
a way satisfactory as a base for my work. 
. .Against the advice of -knowledgeable 
' colleagues, fT long ignored . telecommu- 
nications remote proces3ing and inter- 
active systems — -except for a couple of 
especially suitable- uses — for .the. same 
reason. But in all cases;" enough people 
worked long enough to make the various 
systems pperate iji a satisfactory and ^even 
elegant and fruitful manner. 

Of course, most of us are not inter- 
. ested in the whole computing field. In 
fact, it is now so extensive that it is 
almost impossible to comprehend it. How- 
ever, the math programmihg field is very 
rich in itself and its growth h£s both 
paralleled and been a component part of 
the whole experience of the computing 
field* It had its identifiable beginnings 
at the same time as computers, tl^at is, in 
about 19^17-8. • That was when Dantzig • 
developed the simplex method,, in connection^ 
with planning problems for the U.S. Air 
Force. As witti computers, about .five 
years passed during early developments 
before linear programming began to 
blossom, into a practical technique. A 
quarter of a century has ^passed since ' 
then and> for most of this time,. LP has- 
pushed the capacity of current computers,- ' 
to the limit. • LP and the simplex" method 
are still the foundation stones of the 
'Whole field of math programming and the 
vanguard of OR techniques. 

I first started, working with Georqe 
Dantzig in December, 1952.. I got the ' 
idea^ somehow that I was supposed to build 
ah. automated simplex method and that. the-, 
principle problems ■ were the amount -.of 
arithmetic, maintaining suf f icient; .^re-f / 
jCision, and the large storage required* 
We started , out modestly.-. I tried for 2*5 
rows on 'the old Card-Programmed- 
Caloulator and upped this, to UO when we 
worked out the pjfoduct form of inverse. 
•A year later we were trying to do TOO 
rows on the IBM 701 and, when I. solved 
Alan Manne ' s 101 -row gasoline blending 
^problem, I . thought we had really accom- . 
plished something. VJe went to 25 6 rows 
on tl^e TOU, later 512 and then 1024 rows 
on the IBM: 7090. It -was ten years after I 
.started before we had a reasonably auto- 
matic system . for that size problem, and 
then not always. Furthermore, the data 
processing and service routines out- 
weighed the ^algorithms. I didn't know ■ * , 
what I was getting into in 19.52. 

But -then I suspect neither did George 



Dantzig. ♦ The d-e^^elopment of ittathematical 
programming in breadth, .dep^h, prestige and 
applications during, the -1 960s ^^was fantas-^;* ' 
tic-. ■ We had our .share o^ frustrations — 
decomposition/ in, spite of much initial 
interest extensive software efforts, and a 
later revival", has still not been per- 
fected nor applied on a large-scale; matrix 
generation techniques, with perhaps even 
more intensive efforts , still . have • no 
widely accepted 'theoretical or practical 
basis. This area is now. commanding much .of 
my^ own att'ention. However, we- Have an 
international Mathematical Programming 
Society with its own. Journal and which 
draws a bigger attendance at its symposia . 
than the whole computing community • in the 

'-mid-50s Professor Dantzig ' s "work hSs 
-been recognized by a Presidential award - 
'All this and much more is well known to. ^ 

■ you. 

" Perhaps not-^so well known, is the 
institute where I cim presently working-. 
'It is called the International Institute 
for Applied Systems Analysis or IIASi^. 
JThis represents a further extension of the 
field of operations research *to an "inter- - 
national setting.. It .is not clear' what we 
should call this whole field of which we 
are a part. It is not easy to define 
systems; analysis concisely, .'in . fact one 
program at IIASA is to produce a set of 
definitive .publications for the field . 
Perhaps of more immediate interest *is the 
genesis and make-up of IIASA. It has been 
in formal existence only since late 1972 
but this was preceded by several years of 
planning, negotiations and agreements. , 
Professor Howard Raiffa from Harvard 'was ■ 
its first director. In his speech at the 
• Council .meeting in November 1975, -he said 
the- f ollowing ; ■ -."I believed in the con- 
cept of IIASA when I was.. first introduced 
to the 'idea by McGeorge Bundy back in 
1968. . . •'•Scientif ic detente then as now is 
critically necessary' if we are to have a^ 
sane world'." To what extent political 
detente was a driving force in the for- 
mation of IIASA is not clear to me — my 
first introduction to the Institute was in 
October .197 3 when Dantzig asked me to 
attend one of its organizational con- 
ference^. In any event, it truly is 
international in character with particular 
emphasis on shared sponsorship .and. 
direction by both East and VJest. The U.S. " 
and the U.S.S.R., each contribute equally 
and in by far the largest amounts: There 
are -13 or 14 other countries, who each 
contribute an equal but much' smaller 
amount. This funding 'is not done diriectly 
as government grants btut through Academies 
of Science or equivalent organizations in 
the various countries"! Austria contri- ' 
butes in a special way through .providing 

. facilities at very low or eveff token rates, 
which is in conformity with their ■ policy 
of becoming an international meeting 
ground- The .Institute is housed in th^ 

' refurbished summer palace — oi; "Schloss , as 
it is called — of Maria Theresa, at 



LAxervbiirg about' 10 miles south of Vienna. 
(It is'quite^an experience ^ working . in the 
midst of imperial splendor.) As to its 
mission* I can do no better than, to again 
quote Raiff<M "I "believe ;IIASA has a 
.mission. It milst keep its doors open, so 
that scientists who -have a broad vision of 
. tomorrow *s world can. co'mmuaicate with each 
other. _VJe »cannc>t afford to let these 
doors shut tightly because of some ^ 
:% TBjJHftrneral^ trivial problems. We must 
•learn today how'we can jointly grapple 
.'V* ^- with the stream of equally devastating- 
^'-^r "' -problems that will surel^^r shake our * 

societies' in the not too distant future." 
Raiffa's remarkable performance in 
^.getting the Institute under way has become 
, * almost legendary. He was replaced a year 
ago by Dr. Roger Levien from the RAND 
Corporation-. The chairman remains 
Professor Jermen Gvishiani from the USSR 
Academy of Sciences. 

For the past few months,. I have been- 
working with tt^e Energy Research Program/ 
headed by Professor VJolf Hafele from the - 
Federal Republic of Germany who is also 
deputy director of IIASA. IfSfele was one 
of West. Germany ' s .leading experts in 
nuclear; energy and is well known in the 
energy field. The Energy Program is 
'currently the largest at IIASA with about 
two more years to ~go under present plans. 
But there is not tim4 for more details 
about IIASA's organization and ^operations . 

There* are ^many aspects to .the Energy 
Program but one is the formul'ation, 
implementation, coordination and opera- 
' tion. of a set of models with emphasis on 

strategies for a transition from fossil 
■ to nuclear fuels. At least that is the 
way it started out. -Hafele and Alan 
Manne, with some assistance also from 
^antzig, produced an energy supply model 
in 197i;, It was a dynamic -LP model and 
there have been several variants and sub- 
sidiary models. . It remains 'the center- 
piece of the jnodelling system but many 
' other areas have Had to be investigated 
and models are linder development for some 
of them. These include energy resources, 
energy demands, -long-range economic 
forecasts (the last two being among the ^ 
most difficult) and secondary investment 
requirements. Hafele a^ked ttie to 
, coordinate — he calls it orchestrate — the . 
computerization of the entire modelling 
effort. ' In a practical, s^ense, this turns 
out to involve^ somewhat ■ more than merely 
computer aspects. ?rhen one is trying to 
.coordinate computations ^with a set of 
m'odels variously formulated by Russians # 
Germans, Americans, Austrians," Frenchmen 
and others, one must become involved with / 
concepts^ definitions,' nomenclature, 
feasibility of approach arid so on. r^rie 
must also* pusi^ aside with"^ome diplomacy 
certain formulations and levels of detail 
, '■ which ^a re unsuitable for the overall task. 
Many of the models are LP mdders. 
The Russians, in particular, use LP. to an 
almost unbelievable extent in their 
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national planning and so it is under- 
standable that they "^ake the same approach 
to international -models. XVe also hope to 
have no^t only many component models but ' ♦ * 
versions for at least a few regions of 'tlje 
world with a global model of some kind^ori" 
top. Hafele has misgivings about tRe . 
appropriateness of LP for some of th^se 
"purposes, and so do -I, but there is 
'.clearly much LP work to do in any event* 
It is clear that we will have to 
solve some models^ many times on an Inde- 
pendent basis in order to get started. ' It 
is my hope that ,we may learn, enough this, 
way to be able to circumscribe the ^leaning- 
ful ranges on parameters and device', 
scenario parts which can be combined for 
various cases. We don't 'y^t know how to ■ 
handle the entire system of models in one 
piece so why not cut it up and solve some 
of/ the pieces a number of times to .get ^ 
boundary conditions for the others. Ma'^fbe 
you call this suboptimization but it. is 
better than no solution. However, for this 
to be meaningful, as many of :jthe models -as 
possible must be put into a consistent . " 
framework with respect to style, structure, 
nomenclature, indexing conventions and so' 
on^ c' ' 

However, my main difficulty right now ' 
is that Iidon't have a suitable computer 
even for the pieces. I* have the software., 
and there is much other useful software 
available to us, but-"not the computers. 
Considering the nature of "the place , where 
I'm working, this shouldn't be so. I am ;: 
n'ow. more than a little envious ^f all the 
hardware I've seen in corporate head- 
quarters, investifient houses — yes, and 
universities—which .people .are really just 
playing around' with. I don't want to take 
it away from- them because there are 
beneficial side 'effects — a point I'll re-; 
- turn to at the end. But there \is some- 
thing badly wrong with a world economic 
order that gives its' best tools to those 
who' really don't have an essential need 
and 'd'enies them where they are needed for 
studies with global implications.' 

At the Math Programming S.ymposium in 
Budapest last August, I listened to a- ^ 
Huiigarian give a paper on state plarlning..- 
project he had been^ involve.^ in. They 
worked over -a- year 'and went through all 
kinds of devious methods to solve their 
model. Essentially, it was a big^ GUB * . 
- -.problem- All he n^^ded was a few/ hours 
• with MP*S-III on a. 360/65 or better and the ' 
problem would have been solved. I spent 
some ■ hard years developing GUB in MPS-IIJ. 
Here was someone- who really needed it but- 
couldn't get at it for political /a^nd 
economic reasons.. It strikes 'me that there* 
is also something wrong w£th developing * 
tools and then not making them availablec, 
when they are ner^Ied*. The- money is -spent 
anyway. If the hur.garians .do a -better job 
of planning with our tools, how can. this- 
possibly hurt us? Oh, I'm fully aware of 
the differences between East and. VJest and' 
the .difficulties of normal economic 
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^lationships. Somehow this doesn't seem- 
Very important anymore- . 

^erh^ps you think I^m either; of f-base 

/.lorj 'just^ ri^'ive* I think not. As short as * 

' histbry^of otir field, is, it will be a . 

\shoj^ter time still before enormous hustan^ 
calainifeies begin -to occur. I don't mean 

*^'l5igr*w^s — God for:bid the world becomes 

•::.^e*iBroiled ixi any more gf those," al,though 

^-I could^ point .out a couple of disadvan- 
'teiges. we now face, if it did happen. But 

"*'«:alajnitieS', are already happening. Tens 
or hundreds of thousands of people have 
already sjtarved In 'Bangladesh and middle 
Africa, in j.ust one season. Some people 
now think Jndia is a hopeless case. - 
Closer to.h'ome, the.^ United Kingdojn is a 
catastrophe. ?7ithin living memory, the 
British Dnapire> -was the greatest power on 
earth. It appears that England may soon 
be reduced to a poor country. Even some 
Frenchmen feel 'bad about i£. Here ^t 
home," we have had one oi-'l .crisis-^you 
took it seriously if you were living in 
Boston at "the time. 'There will be ijiore . 
It appeeurs' likely that OPEC oil prices 
will -go to $15.00 per barrel this- month.* 
That is not yet, a Catastrophe, for the 
U.S. but prices will go higher and higher 
unless^ some drastic changes are made in 
our economic pblicies---that is.. Project . 
Independence and much more must become a , 
reality r ^ . ' : 

Eduard Teller visited IIASA last: . 
month and gave a most wonderful ,talk..on „ 
alternate sources of energy* said he", 

didn't know whether the really serious 
problems df the world could be solved 
but he was sure of one th'ing — the^ 
couldn't be -solved without plenty of * 
energy available. Furthermore,' much of it 
has to be in a form which is .usable in 
small doses in m^ny places. For example, 
it would do no. good tp put a thousand 
megawatt reactor in' an ^indeve loped " 

. country. -'.They simply don't have the ■ 
wires to -carry thfe- electricity away or to 
distribute it tp all tjie places it^is ^ ■ 
neede"d. The only form we presently have 
whiph is usable this 'way is petroleum. 
But if the U.S.- keeps gobbling up ^5% 

-of '.the world's production, the price^ will 
keep going up putting it out of reach of 
many na.tions who -can barely pay for . what 
they use nowl . Furthermore, we only • ^' 
hasten the day and postpone the prepara- 

-^ibn- for it when we will begin to run 
short ourselves. And^ imless' we drasti- 
cally change our style of living, that 
will be^a M^ata strophe here. 

. I bring these things to your atten- 

' t;ion 'to indicate the kind^ o f problems 
the world mdst be; about solving- — and 

immediately w I will add to Teller's 
remark ^d say. t^he 'rpally serious problems 
Oan't.be solved without effective use of 
computers on a wide scale. . Not for 
keeping accounts, or * printing electric *♦ 

'bills and bank, statements, or even for 
simulating the aerodynamic qualities of a 

:new airfoil, .but for systeirjs analysis or 



whatever you wish to call it, seriously 
and 'expertly applied. That may not be, 
sufficient but it is necessary^ Further- 
more, the studies, plannirrg and decisions. 
mua*t be intelrnational in scope and that • 
jnean*s across East-Kest boundaries^, among 
"others. "^Othei^ise ,* either they will be • / 
ineffective or there will be -counterplays 
, which come to the same -fehirng. .And the. - ' 
world. has run out. of time for playing . such 
. , games. . . , ' < " ' ' . ^ / 

, ,i . was^talking to .e^ senior . French . - ^ 
analyst the day after Teller's talk and we^ 
,got to philosophizing a bit. . I saidv I 
thought there would be such dislocations in 
the world during the^" next quarter century 
"'t^at no one could predict how the global . 
economic, order woul'd ^change . ^ He agreed 
and even gave . some examples from history - 
I then added that r thought a catastrophe 
in 'one area would be bound to have an 
ef feet ,on the rest' of the world. Here he* 
/-shook 'his head.. No, he said, if a million 
• people died in India or Africa, some • 
people in Paris ^ould send a little money 
to the Post Office for a relief* fund but 
•otherwise life would go on just the Scime. 
'No one would really care.' Did the* French 

care when Spain was living in poverty?' 
-No> it ^was a f ine place for cheap vaca- 
tions^ . ^ - ,■ 

•Still, 1 must believe that the cumu- . 
JL^tive effect of a series of calamities 
will be widespread^ the more 'so' as they 
.begin to be chronic rather *^than e.xcep- . 
tiorial. The 'decline 'of British power, for 
"example, is probably a -tragedy of greater 
consequence than starvation in some un- 
developed, country . . Why? Because, whether 
the British were liked or riot, -they nearly 
always left things- in a better state than 
they found them!. If they" can not now., 
^manage their own affairs, the World has. 
, 'lost one force for improvement and 'gained 
still another problem. ' At the least, .it' 
should be a sobering .T.esson to us.t ; 

' • J. think, all of us, even including 
knowledgeable scientists . and' high officials 
'of state, 'have a. .tendency to disbelieve 
what seeJhs ^monstrous and -new. J^e think of 
such thing;s abstractly . unless and until' * 
^ they come' down to affecting our daily - 
lives. V7e might »do "well -to remember Marre 
Antoinette^ and the -Romanovs. "Optimism and,^ 
courage are great virtues- but^ they lie 
very . cl^ose' ,to foolhardiness. x It is /- 
commendable to be astute -and shrewd in 
' business but this 'should! be -guided by^ ' ^ 
vision.' ^ ■ ' ' . 

^/'The basic outline o;f approachi*ng 
world problems^. was recognized twenty years . 
agd. The -late 'J-D- Williams gave some 
very easily ' unde^rstood-, dramatized papers, 
unfortunat^ely probably heard or read by 
very few people. I attended' a . series of 
seminars at the American Management * -t 
Association in 1951 3bhd 6ne lecture was a 
partly humorous but highly cc5hvincing 
dramatization of the alarming positive 
rates' of change in a variety of areas.- 
(fl regret that 1 have forgotten the 
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speaker's name; he was from Washington.) 
: He 'presented th-is regularly to a variety 
of business, and government gatherings. 
Perhaps he made it too easy to laugh. 

History tells us over and over" that 
no arramgeroer^t is permanent. , It is 
foolish to Jbelieve that because we are 
strong we can continue to consume far 'more 
than our proportionate- share of the world's 
goods* If it were only a matter- of others 
attaining' the degree of affluence we enjoy, 
then .it might be considered 'reasonable for 
the technological leader to ha^^e the most. 
This has been pretty much our attitude. 
V7estern Europe has in fact gone a long way 
toward catching up though their per capita 
-^consumption /of energy, • for. example , ' is 
slightly less than half what ours is. In 
other areas there is a -discrepancy of an 
order. of magnitude and, in some cases, 
almost two orders. Some of these 'areas 
must increase energy consumption 'just in 
order to provide food. But the ready . 
supply of easily distributed energy forms 
is becbming limited. Our own strength' / 
and prosperity depend on.hi^hl^y developed 
uses' of these same forms. We need not 
moralize but only consider being placed 
in competition with people who must have • / 
some of what we want simply to survive. 
In only another 25 years, the .population 
of ■ the .world will have increased another i 
5055 ^ that, is, by more people than were 
living when I was born. If any substan- . 
tial number of these people improve their 
' living standard by even 105S to 205S — 
which is almost nothing — the" strain- on 
world. Capacities "wi 11 be fantastic. 

Let me put it another way. There 
have been extensive , serious and competent 
studies of world problems for a number of 
years. Most of these are based oh 1967; 
-occasionally .1970^ figures;." They talk 
about 5 year^ 10 year or '25 year pro- 
.jections, with a few attempts at. long- 
term forecasts for^p 50 yeairs or longer. 
But it is this minute very^ close to 1977. 
More than five ye^ars have passed since « 
1970 nearly ten since 19'67, and the " 
leaves of the calendar keep flipping". .-It 
will soon be 1980 and then 1985. Already" , 
world population will .be at about 5 
billion. When one measures this against 
tlie time for building an' industrial com- 
plex, improving a transportation system, 
resolving^ political issues, achieving 
international accord for even modest 
pro jectsr-well: it is clear that many ' 
babies born this very day are doomed 
already.* There- is no such thing as 
opportunity to' improve their lot or free- 
dom to. choose .their carreer. 

So what can we do? Over the past two 
decadeS/ we have developed very powerful 
tools for , analysis and planning. A great 
de$n of. hardware and software now works 
and works well. Modelling techniques, 
though far from perfect, are well advanced. 
We are now in. a position technically to 
make real inroads oh world problems, to 
undertake the kind of work which 



originally motivated the development of 
large computers , and planning and ana- ^ 
.lytical techniques. But we have not -yet 
learned* to coordinate their use consis- 
tently on a broad scale ^ noif to convince 
the actual decision— makers that they can 
piace*^considerable reliance on results. J 

We must find ways to make the ^use of 
our marvellous tools more, effective. 
First of all, this means use of inter- 
active systems on- both large and small 
computers, tied ^together with networks in. 
a consistent manner. All these things^, 
exist, piecemeal, and are in daily,- 
reliable use. We have the. technology but 
we lack .coordination a^d singleness , of 
purpose. In the West, this is due to 
business competition-^including the 
iiTiiyersities and research centers which 
are nothing more or less than large 
businesses.'. In Europe? the discontinue- *. 
ities between traditional nations and 
peoples add .further dichotomies. Between 
the free market .countries and the 
planr\ed-economy bide , fundamental policy 
differences exist'. Comparing all these 
more or less- developed countries with 
tinderdeveloped countries, one finds that 
the latter don't even understand the game. 
They know something of the basic tools but 
very .little about motives, incentives, and 
other driving forces. ' Of course, there 
are individual exceptions. 

Usina coitrouters with common conventiDns 
for analytical work can 'be a very strong 
unifying force. No doubr much of the work - 
of this kind has been of small value but . 
if it brings diverse.peoples together a 
little,' this is in itself an accomplish-. - 
nent. Moreover, some of the efforts are 
certainly first rate and fruil^ful. The 
more wi<3ely they can be understood and 
appreciated, the more quickly we can make • 
: effective use of them on a scale appro- . 
priate to the. g3^obal problems we are- all;^ 
facing. . In short!, .our discipline needs' 
mOTe. stahdardization. 

Of .course, standards mean different: - 
things, in different contexts. Some are .. 
essentially a legalization of what is 
already considered good practice, such as 
state commissions and examining boards 
over a wide range ,from barbers to lawyers • 
and doctors. For 'more technically com- 
plicated areas or when massive or tedious 
data must be gathered, we have such in- 
stitutions as the National Bureau of 
Standards and the American Standards 
Association. But with apologies to our 
hosts, these approaches do not apply to 
the present discussion. 

De facto standards — usually due to 
economic forces — are often- among the most 
effective. Several years ago there was 
much discussion of electronic modes of 
recording on magnetic tape. Many people 
claimed IBM's methods were not very good, 
i don't know whether . they were or not but 
they 'worked and, to stay in business, 
other manufacturers had. to be compatible 
with them.. Now one can put a tape in his 
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brief-case, travel^ over *^a large part of 
the world and have the tape read by a 
computer in some remote place. This is a 
^great advantage. I think no one would 
claim today that the MPS/360 input formats 
are very good, but for the same reasons, 
they are an effective* standard. As. a 

' result, problems can- be shipped around and 
solved on various computers. ' Furthermore, 
it only takes one' word to describe the 
format, another great advantage. 
^ ' Unfortunately, economic forces can 
aiso work the other way. One sometimes 

"suspects that economists, operations 
researchers, and' other system analysts — 
not to' speak of software designers — are 
just as happy if their techniques cannot 
be measured and compared too clearly and 
precisely, another result of intense 
competition. But competition won't. solve 
today's problems; collaboration is much 
more to the purpose. ^ 

In the task I'm presently engaged in, 
it is necessary to devise some sort of 
standards and,. one way pr another, to en- 
±orce their use,* To d6-^is', they must be 
logically consistent and explainable — 
-which is perhaps the best kind of stand- 
ardization." I have discovered that this 
is not -an easy task but, more importantly, 
the a1;teropt leads to ^considerable clarifi- 
cation of fundamental ideas - Let me 
illustrate this with the problem of— rden- 

' tifief s. 

' As most or* all of you ,know, an LP 
model has- got to have unique row and 
column identifiers and, in a practical 
sense, these ar^ limited, to 8- characters 
with only 36 to 38 graphics available. 
Since- some formulations require five or 

' six indices, one. must be extremely frugal 
with, encoding schemes. This forces one 
to study very carefully just what it is 
that one is trying to represent and this, 
in tiim, leads' to some rather deep con- 
siderations.. One runs into such problems- 
as what constitute substcmces, what can 
be called processes, is there a funda- 

- mental distinction between primary and 
secondary conversions, what are capac- 
ities and how are they related toother 
variables? What is capital and what is ■ 
labor? These last two gave me consider- 
aJDle trouble but finally I thought I had 
them.fcorreq^y classified. I spent most 
of tSiSimday^ rBadLng the encyclopaedia to 
see. if my conclusions were correct and 

' f bund^that essentially they were. J. ' 
learned much more which\ I would never have 

••comprehended if I hadn't been trying to^ ' 
figure out how. to automate matrix"" and re- 
port-generation and to make them con- 
sistent over a range of models. Some 
economists would do well to go through the 
same drill to clarify their own thi&iking. 

Once the manipulation of the models 
is conveniently automated, we can then 
begin .paying attention tp the really im- 
portant questions — the validity of data 
and its meaning the effect of variations / 
i.e. sensitivity €malysis, the effect of * 



changes in hypotheses and. h.ow this relates, 
to model structure and its projection in 
•reality. It must be possible to^ sit- down, 
'at a console and, during a morning, get; 
solutions to several variants of a model,- 
with hxaman interaction with the computer 
quickly and cryptically communicated. This 
means int-eractive systems and, in most 
situations — certainly ours — it means 
reliable telecommunication facilities. 
Only then can .we begin to get some- feel 
for the possible behavior of our compli- 
cated world. ■ * . 

Thus the computer, used as an analy- 
tical tool—in fact, almost as a ^ 
colleague— will be not only a computational 
engine but a unifying and standardizing 
force. There will still be plenty of 
differences of opinions but the issues will 
be clearer and facts — as nearly as we can 
approxiiria^e them — will stand out. . 

But all this will happen only if those 
of us who are skilled in and mo'tivated by 
the effective use of computers begin to 
assume leadership- The world is buried in 
scientific and technical journals and there 
i*s jio end to ever-multiplying, complexity * 
of thought and confusion of detail. ^ It is 
a*lmost a sickness. The need now is *to take 
hold of all our skills, tools, .and proven 
techniques and mobilize them to the best 
of our abilijiy in order to clafdify issues., 
influence me'aningful decisions, promote 
rational reoperation, and continually 
sharpen our perceptions. We have nothing 
to lose and we,migh€ just discover a 
world order that is workable. ;5. 
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IMPLEMENTATION AND APPLICATION 

A NESp© DECOMPOSITION ALGORITHM* 
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ABSTRACT 



This, paper considers, a nested decomposition algorithm* for multi-stage linear, 
ptograms^. With tlie" staircase structure,^ Computational aspects of the algorithm essential 
to an efficient implementation are discussed- Experience with using experimental codes 
of. the algorithm on problems arising from real applications/ such as energy models, 
engineering design, and dynamic traffic control iis presented. ' It is observed that nested 
decomposition caoi be an efficient techniqu^> for large-scale' systems. 



« ♦work . performed under the auspices of the ERDA- 



1. The Staircase Algorithm ^.. 

we consider the linear programming 
problem of minimizing 



as follows: ^ 

aubjttct to 



(21 
(3) 
(4) 



■abj«cc to 



*1 *1 - \ 



■t-1 *t.l 



(1) 



In L3] and [73, the s£aircase iL'go- 
rithm is developed from an application of 
nested decomposition to (1) . ^ Using this . 
algorithm/ the x>riginal problem is re- 
placed by a sequence of smallisr, indepen- 
dent subproblems coordinated by primal 
(proposals)^ a[i^ dual (prices) information' 
' in the sensie of Dantzig and' Wolfe [21 • 

A cycle of the Staircase algorithm, , 
• indexed by. Tc, consists, of T suibprobleras 

denoted by SP^* t =1,.*.., T, and defined. 



. * ; T:h e var i ou s t er ms in* the s ubpr ob 1 ems 
. are "defined recursively 5 as follows, pre- - 
ceded by their dimensions, A prime denotes 
a transpose. 



scalar: p. 



•calar: 



, (t - 2, 



T) 



. (t - 2 T) 



} 



(6) 



^For SP^. delete the terms involving \£ and 
also equation (4) -"""For SP^, delete the 



terra n|^j^ B^ from (2) 
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23 



Ik 



ft - 2. 



T) 



(75 



1 < tct 



n *^ -2 



rt - 2. 



T) 



(8) 



1 ^'^^i . "^^ i» th« v«et:or of doAl v»ri«bl«« for (3). 
•c£lWsi ia th« dtiial vmrlAbI* for (4). 

SP is read as subproblem t at cyc^e k. 
SP denotes subproblem t at an unspecified 

cycle. The (1 + m^) ^ ^ vector (p^, q^) 

"k k 

is called^a proposal from SP^_^ to SP^, 

k ^ 

The prices for SP^ are given, by 

/'k ■ k • 
.<-t+l' ^t+1^- 

The three phases of the Staircase 
algorithm are summarized below. 



Phase 1: 



Step 



(i) : -Set t 1. 



Step' ' (ii)t *Start with . an artificial- basis 



for SP, 



Set the objective 



to be tti.e sum of the infeasi- 

bilities in SP. . Use the 

* . t ■ 

Phase 2 procedure to . solve 
the subsystem LJSP^ , . - • ,SP^1- * 

If the optimal value of the 
objective > 6, stop: . problem 
is infeasibie.: Otherwise ^ go 
to 'Step ■ (iii); -if '.t' < -Tr ' go to -. 
P-hise--2 : if t ■=- T.:!! . ... \\ ; * 



Step (iii) t Generate -a proposal for SP 



t+l" 

Set. t ,= t + .1.. Go to Step (ii)- 



Phase 2; 



Step ' (i)i Set k = 1- 
Step (ii)r Set t = T. 

Step (iii) : Solve" SP . If t < T*, send a 

. ^ . proposal (if any) to ^^^.^^r 

vif > ^1 , and - subproblem is 

bounded, s.end'prices to SF^'_.^> 

•otherwise, go ..to Step (ii) ; if 
* t ='T 'and subp^r^lem * is- un- 
■ . . bounded, stop: problem is 
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unbounded. 
Step (v) . 



If 't = 1,. go to 



Step (iv) 
Step (v) 



Set t = t - 1. 
Step (iii) . 



Return to 



Phase 3^ : 
Step (i) : 

Step (ii) : 



Te^ for optimality: 

= Z d^. If optimal, 

go to Phase 3. Otherwise, 
set k = k + 1^ go to Step (ii). 



Set t - T, y^ = x^/ compute 
d^ - A^y^- . ^ . 



Solve SY^. 



Set t. - -t - 1., 

If t = .1, stop. Otherwise 
compute- d^ - y^. Return 

to Step (ii) . 



A flow diagram of 
-given i^ Figure -1/ 



^ ....... 



2 - I mp 1 em entaL t i on 

The computational efficiency of an 
implementation of the Staircase algorithm 
depends essentially on the . following three 
aspects: .'■ 

Ci) data structure:' /^This .pre- 
scribes, the amount of data 
required to define a subprob- 
lem , and- the amount of data 
transfer required to update 
a subproblem* • - 

• -*V':. (ii)- solving * a subproblem:. ' This-.; * ^ 
,is~ how the subpfobiems are 'to 
' .be solved as linear programs - 

(iii) coordinating information: 

, This, is how prices are incor- . 
"pqrated^ and how propoisals are 
" .generated in a. s.ubproblem. ■ 



= D«fln* for t - - 1. 



* subject to * Cr^u^ - 



Por.SYj^ d«l«t« frmm involving. w^. 
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For our e^cperxmental cpdesv 
the guide-line of making- "f jail .*us^'*i 
vance simplex techniques ' in linear 
ming for solving the subproljle^rig .( 
[lia, [14]). Efficient, designs for' 
(iii) compatible with^ucli a scheme Az^e 
then identified.* Although we have "hGen' • \ 
limited by th^ scope of this research tb * ' 
give certain considerations t^' convenience 
of programming, th^* underlying -concepts are 
nonetheless general and should* b<e applica- 
ble to eyen the most •sophisticated imple- .'' 
mentation. 



3, Data 'Structure 

• <;r^ Large scale problems are usually very 
sparse. . The density of nonzero entries in " 
the constraint 'matriT?^ is typically' Tess 
than one percent [ll- Although the. nonzero 
entries concentrate in.blocks for staircase 
structures , the subproblems defined on 
these blocks should still be sparse. For 
example an 0.1% density in a 10-period 
1,000 X 10,000 -(row by column) problem inv- 
plies an average density of 0*,526% for the., 
non-zero blocks. " Therefore, the- subproblem 
data should be stored in packed form* Many 
.schemes are availaJl^le for storing only the. 
nonzero elements (see e.g. [83, [121) • 
Since our algorithm consists mainly of col- 
umn operations, we use a column packing 
scheme. The ndnzeros ^"re packed in a vec- 
tor (one-dimeftsional array), by column or- 
der. A second: vector of the. same length 
contains the row indices of the correspond- 
ing entries in the first vector. A third- 
vector gives .for ea'ch column the position 
of its first entry, in the other two vec7 
*tors.. In the Staircase algorithm, the sub- 
probiems ^ are- modified by the- addition- of. 
propdsais, which form new columns in the 
Constraint matr ices .. With t^he- above . scjreme,.* 
'appending - a new. colusan is done by simple . . * * 
extension of the three vectors. 

In general # the data fbr each. subprob- 
lem C2U1 be classified as follows- 

(i) constraint type data, 

(ii) right-hahd-side data^ 

(iii) basis data/ . * 

CivO constrain^^iatrix data, 



proposal data* 



fro 
and 



The first two classes remain unchanged 
one cycle to the next during .Phase 1 
hase 2. They are modified in Phase 3. 



The amount of basis data that needs to be 
.stored depends on -the subproblem solution 
'^5roce^u^. In our case, these are simply 
^n^^^^ex^ vectors identifying a basis* Class 
; ^r^^(iV) *N5i%n^i^^« 'of data* .for. ^the current .9o.n- _ 

.'^straiiit matrix, while (v) contains data 
.* . ^for^ the new columns to^^be added to form a 
new subproblem 'in the fol^wing cycle. 
, * TWie necessity to differ ent:iat^ between (iv) ^ 
and (v) depends on the relative efficiency 
in concatenating data records, which* is :in* 
turn determined by the data storage device 
and mode^of V^ata transfer being used., 

' ^ 'Subproblem data are stored out of 
^core. Regions 'in out-of-core memory are 
designated .for the various classes of data. 
Each' region is subdivided into T subre- 
'gions, .correspondiiig to the number of pe- 
riods^ in the problem. To solve a subprob- ~ 

em, data fr6m the Appropriate subregions 
3^e rfead into the work region* (scratch 
space) in/'core. Oatputs from the subprob- 
lem. are writt^ into the appropriate sub- 
regions as data for later cycles. The 
data^ flow for a subproljlem is illustrated 
•in Figure 2, 



' 4\ Solving a Subproblem • 

To solve the subproblems efficiently 
as linear program^, we use a product-form-, 
inverse (pri) revised simplex routine with 
an inversion subroutine designed to pro-' 
duce sparse representa,ti<)ns of the basis 
inverse. This routine is based on LPMl, 
an in-core" LP code, written by J. .Tomlin in 
; 1970. [143. ' ■ 



5. " Cobrdinatifiq Information ' . 

J ; The typical s.ubproblenr- cart be arranged- 
in^ the ' f orm ^Hown in" ;Figure 3 where — de-" 
notes a '-no n-bi nding row. No pivot opera- 
tion is'^^o be performed on such a row. 
^Apaxt^ from being_consistent with the data 
structure described in Section 3, this full 
subproblem formulation has the advantage 
of allowing (aV dual prices'^ to be incor- 
porated into the •'Objective function with- 
out any change in origi^nal data and (b) 
primal proposals to be inferred from the 
updated' right-hand-sides of the noh -binding 
rows .To ■ show .--this, we use the simplifying 
notatxon- in Table 1. ' 

Now, suppose we maintain a basis for 
the full subproblem including the non- . 
binding rows.* Such a basis will have the 
following form: 
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A - 



1 i 

0 k 

0 



0 
0 



A 

3 I 



(9) 



where the entry 1 and the identity sub- 
matrix I indicate that the slacks^of the 
non-binding constraints are alv^ys basic 
since no pivoting is dojie on such rows. 
The. basis inverse is then 



Suppose we now require rr to satisfy 



TTM - (Ir 0, -"r^t+l^ 



m 



t+1 



m 



so that" 



(12) 



components 



1 -a^-^ o' 



AA- 

LO -BA 



(10) 



The objective function for SP^» as given 

by (2) is (dropping the superscript Ic for 
convenience) , 



TT (1, , 0, 



t+1 



, A-l 
)M 



(1, 0, -TT^^,) 



1 

:0 
0 



0 
I 



A A^A-1 _ * 



which is to be minimized • In the present 
notation, ^ " 



(c - 'rT^+iB)x 



(11) ■ 



We argue that it is not desirable to set 
up 2^ explicitly, ^Fits-t, a direct multi- 

. plication of ^^^^^^ ^ would make it nec- 

essary to distinguish between data in A 
: and This is cumbersone since the ma-, 

trix.data are packed ,'_cblumn" by .column! - 
Secondly, to update the - obijective, we: heed 
to store the original e sepaiJately- since 
.this is used every time we compute a new' ■ - 
2. • Finally, updating lyequires the accom- 

modation of new Tionzeros » This cannot be 
done efficiently in the data structure 
being used. All^ these complications can 
be circumvented as follows. Recall that 
without the tt. . - B term in the objective, 

^the simplex multiplier tt for the basis-^M'' 
would, satisfy. 

m components 



tA « (1, 0, 



0) 



This gives a reduced cost of 1 to s^^ the 

slack variable in the objective row which ■ 
is always basic since the objective row is 
noxibinding, and zero reduced costs to the 
other basic variables. 



Letting = (c - tt^_^-^)^ , we have 
t t+l * ■ 

TT = (1, -n^/ "^t+l' and the reduced costs 

are- 



C » TTM » C - (C. - ""t+l^^^ ^A - TT 



t-HL 



s^ that we are effectively using z^. 

Kote that (12) implies that the re- 
duced costs of the slacks variables in the 
last m^_^^. rows are precisely -n^_^^. These 

variables are always basic since the cor- 
respo'nding rows are nonbinding.- Therefore^, 
we can interpret (12) as ■ the setting of 
prices on the inputs and outputs -described 
by the matrix B. 

Next, consider the updated right-hand- 
side given by (15) where x gives the values 
of* the basic variables /in the current ex- 

treme point soLution x of S^^- non- , 

basic variables in x are, of course, -at 
zero value- 



^This is the backward, transformation or 
BTRAN step in the revised simplex method. 
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1 











AA " 


* 




"ft 





t 








a m 






A 




A 




\ 


3^ 
..3. 




1^-^ d^" 




AA 

-Bx 





_ _^ If .A*__5 _a" A^_^_ 0., ._th^nisP^.._is_. 

unbounded^ from, below, with 

jL - (0 0, I, 0 0. <-A*),. 0 0) 

po«itiop;j poiition - 
. basic in i^^ 



(19) 



(15) 

Therefore, according to (5), (6), and (15) > 
-a .proposal corresponding to x is given by " 



(16) 



which, apart from the .sign, is part of the 
updated r idht-hand-side 9- 

^ Proposals corresponding to extreme 
ray solutions of SP^ can be generated is 
follows. Suppose a column in M, say 







'ex." 






m ■■ 
















J3. 



is priced out to be the pivot .column. Its' 
trans format ion* with respec^to the basis 

ft. is . '". - . ' ^ ■ ' 











"i 


AA-l 

-CA 


0 




r s n 

c . 


N » 




t 

A-1 

- M N 


m 


0 




0 








.^3- 






. 0. 


AA-l 
-BA 


I- 







c - 



B - 



^^A= 

A^ 



as a homogeneous solution on the extreme 
ray (cf. Theorem 2.1^ p. 35 in [l3l)-\ By' 
definition, the extreme ray is simply 
x^/(e-x^) where e = (1 I). Observe 

that by. (5), r^TV(17) and (18), the pro- 
posal corresponding to" is 



5^ 




"3 J 



(19) 



;(17) 



which is part of the transformed column . 
The proposa-^l corresponding to the extreme 
ray can, of course, be 'obtained from (191 
.by scaling with l/(e-x^) . However,.* this . is 

not necessary and we may simply send the 

k+1 ' • ■ ■ » 

proposal in (19) to "SP^^^- This is equiva- 
lent to an implicit scaling of the corre- 

^ )c+l . • • 

spondiTng X. variable in SP^^^ by e-x^, hence 

ther§ is ho loss of generality. .- ' ' 

Finally, to determine whether a pro- 



^ posal is profitable to SP 



k+1 



according to 



the^' prices we ha^^to test whether^; 



^t Tt+l <^ 



"t 



= < 



for extreme . point proposal's ; 



Vfor extreme ray, proposals. 



This second case is always sat^isfied since 
is unbounded from below. Hence, no • 

computation is required. For the' first 
case* we compute ^ , 



*This is the' forward transformation or 
FTRAN step in the revised simplex method. 



^In the Staircase algorithm,, "proposal" and 
"profitable proposal" are used synbriymous- 
ly, so that strictly speaking, the vector 

. defined in .(16) is a proposal onl/ if it 
passes the .test. * 
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t+1 



(20). 



whicb is essentially the evaluation of an 
inner product . 

We have shown that as the result of / 
ajJplying the revised simplex procedure to . . 
the full subproblem 



tainimize 



cx 



siabjcct to C_3ac' ■ d 



X 0 



.(21) 



(a) the dual information J^^^^^ is efficient- 
ly, utilized as part of the prices tt 
for (21) ; 



(b) 



the primal information [^] is effi- 

■ ■ . Q 
ciently generated, being a by-product 
of ri-ght-hdnd-side updating in the 
case of extreme points, and forward 
transformation of a pivot column in 
the case of extreme rays. 



■6. Refinements ■ . 

Within the basic framework summarized 
in Section 1, many refinements of the 
Staircase algorithm are possiblie. Such 
modifications, are called computational 
strategies. Some are designed to acceler- 
ate convergence; others to reduce, storage 
requirements and the amount of ' data trans- 
mission. . Most of them are motivated by 
h'euristics and have to be vaTidated dnpir- 
ically. Moreover, good strategies for one 
class of problems may turn out to be poor 
ones for another. Therefore, it is im- 
portant to parametrize such refinements so 
that a process of fine tuning is possible 
for any given class of problems. We iden- 
tify„three important strategies here^ For 
more detail, the reader is referred to [3]. 

(a) Deigree of Decomposition: 

■ ■ '■ ■ 

For a given amount of core . storage, 
it is often feasible to vary the number of 
subproblems by grouping two or more as a 
single stage. A higher' degree-of decompo- 
sitioTi gives smaller subproblems which are 
easier to solve, but is' likely. to require 
more interactive adjustments before 



obtaining an_'op^ip*i_ co^rdinaJ:ion ■2tmong 

" 't he s Spr ob 1 emsT I ni t i^ 1 exp¥r i enc'e ~^wi't h 
the Staircase algorithm [3] suggests that 
whenever nested decomposition algorithms 
are ■ intended for routine applications on 'a 
particular class of problems with a fixed 
amount of core storage^ one should prede- 
termine^empirically a good strategy for the 
degree of decomposition. 

(b) Multi-proposal Generation: 

By generating (when possible) more 
than one proposal from each subproblem, 
convergence may be accelerated. However, 
if too many proposals are transmitted in a 
cycle, the inferior ones, which may never 
become useful, 'simply cause an urfn&essary 
increase in ^he size of a subproblem.*. 
Moreover, proposals that are too.si^lair 
may g^ive riSe to numerical instability. 
Therefore, a heuristic procedure is re- • 
quiried to select a Timited number of pro- 
' posals. A limit of five provided good re- 
sults in most cases we encountered. 

(c) Proposal Purg'ing: 

AS proposals are introduced, the grow- 
. ing size of a -subproblem may cause diffi- 
culties for in-tcore storage. As far as the 

■ ' k • 

optimization of SP is. concerned, the .only.. 

., ■ V ■ ■ : t ; • ■ , .. ^ 

proposals . that need be kept are the cur- 
rently basic ones (for feasibility) and the 
latest profitable ones (for improvement). 
All others could be dropped as th€ty will 
be 'generated again- if and when they becdme 
prof i table, on later cycles. We use a 
scheme to purge as maAy non-basic proposals 
as necessary to keep the subproblem size 
within limits determined by core availabil- 
ity. However, a modification of Phase 3 
is also required to allow for proposal 
purging. . - - . 



7. The Experimental Codes " , 

.-Two experimental codes, natned SC73 and 
SC74 have, been written in FORTRAN. . Input 
data „ is in standard MPS format plus .a sec- 
tion on information characterizing a pat- ■ 
tern of decomposition, except for the data 
handling features. The two codes are iden- 
tical. ' " 1 

SC73'runs on an IBM 360/91 -at the 
Stanford Linear Acceleration Center. *It 
requires approximately 20 OK bytes of core 
storage, when ^dimensioned for problemis with 
a maiximum of 20 periods, >each huivirig up to 
500 rows (as a full subproblem) and 3000 
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nonzero coefficients (including prppbsals) * 
^Secondary ^3 tor ag<^^^^^ 

'track «• 7294 bytes) of IBM type 2314 mag- 
netic disk* Data transmission iS by di- 
rect -^ccess I/O using variables length re- 
cords with a block size of 7294 bytes. 

SC74 runs on the CDC 6000 and 7000 
series computers. It requires approxi- 
mately 35K words of SCM core storage when- 
' dimensioned for problems with a maximum of 
10 periods, each having up to ^00 rows (as 
a full subproblem) and 6000 nonzero co- 
efficients (including proposa^ls) . Second- 
ary storage, requires 170K words of ECS 
(extended core storage) or LCM (large core 
memory) . Block t>-ransf ep of data between 
SCM and LCM is used. 

\. \ ■:• 
For comparison with a direct simplex 
approach, we used MPS/360 and LPMl L 14] , 
the latter being the simplex procedure 
used in fiC73 and SC74. Roughly the same 
storage configuration is. used in each 
C9mparative run* - 

8. Experience wi'th Applications 

[ ■ 

This section presents computational 
experience with t^ie successful application 
of the. Staircase algorithm to three classes 
of multi-stage linear programs. In each 
case, we desciribe the nature' of the pro-* 
" blem briefly and summarize the performance' - 
of- the Staircase algorithm' as compared to 
a dire^^t simplex approach. ' \ * 

(a J Optimal Design of Multi-stage 
Structures [4]t ^ . 

The probl^ is to design multi-stage 
planar trusses for minimal weight over, a 
class 'of feasible member sizes and config- 
urations. The design is based on limit 
analysis subject to a single set "of loads. 
The Variables x^ represent forces in mem-; ■ 

bers of. stage t "in the truss*. The equi- 
librium conditions for stage t are ex- 
pressed through while B represents- the 

t ■ " t 

coupling between "stage t and t + The 
'external loads are given in d^- Typically, 

these problems have a largg number of col- 
umns iij proportion to th^ number of con- 
istraints. . Therefore, the* efficiency of 
the Staircase algorithm could" be. due part- 
ly to a partial-pricing efjEect.^ See 
Table 2. ' . *- ; 



(b) Dynamic energy model [5]: 



These test problems are derived from 
a staircase versiofx of Manne's model of q. 
.U.S. options for a ^-transition from oij. and 
gets to synthetic fuels [9l. They. seek 
minimum-cost strategies to meet future 
energy demands under a series o£ alterna- 
tive scenarios^ The latter depends'"oh 
estimates of the remaining quantities Qt^. 
domestic oil and gafe resources, and • the 
. technical and envirgnmental feasibility of. 
new methods for synthetic fuel "production. 
The variables are production capacities- 
and investment r in the energy -sectors ^ " The 
single-period constraints exert^bounds on 
new capacity introduction rates .and relate* 
production to final demands in energy but- 
put.^ The dynamic constraints relat^ "capac- 
ity inventories and investment, model the 
nuclear cycle, and exert bounds on. cumula- 
tive resource e3ttraction. . . 

The model has 15 periods representing 
five-year intervals from 1970 to 2045. 
Hov^everv-a four-period decomposition i^ 
:used for reasons explained in section. 6a- 
*All new technologies are allowed in pro-' 
blem 4A (see Table 3) while^ most of them 
are suppressed in probJ.em 2a - The per- 
formance of the "direct simplex approach 
(LPMl) reflects this variation in complex- 
ity. Whereas , -^by decomposition , such 
effects .are "felt" by-each subproblem' from, 
the start. This may explain why SC74'^took 
roughly the same amount of time for, all 
four problems. , . 

(c) Dynamic Tifaffic Assignment [6li 

In "the Merchant and ^emhau^er model 
of dynamic traffic assignment Cl03/*a 
-txaffic network is represented by a di- 
. r^eicted graph. One of the nodes is desig- 
:nated as the destination. The planning, 
horizon is divided into a finite number of 
discrete time <5eriods. For each time per- 
iod, external, inputs are allowed at any 
node' except the destination-, -For each arc, 
there is an exit funbt ion which relates the 
amount of traffic entering and leaving the 
arc during a ..time period. Congestion. is''^ >- 
. modeled by assuming the exit , functions to"' 
be nondecr easing, continuous/ piecewise 
^inear "and concave. The problem is to find 
the feasible traffic flow which minimizes . - 
"^he total amount of traffic over the plan- 
ning horii^on. *■ •. 
^ ' . '■ • . . • ' ' 

^ Here the; unknowns aaf^e the amount of. . ' . 
traffic .in each arc .in each time period. 
They are transformed ^to convex comtJinations 
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^lAtear exit functiiQhs. Therefore, the 
variables are the interpolation weights 

for these combinations. 'Th6 single-period 
constraints are the iflow balance equations 
for the nodes. The dynamic ^constrain'ts 
are the flow balance equations for the' 
arcs. . . 



The Staircase algorithm is us6d as 

part of a hybrid algorittwn [6l for this 

class. of problems. See Table 4.. 

■^"^ . - 

\. 

9- . ' Remar>cs - " ■ " 

■ • * 

It has been observed in [3] that re-' 
lative to a direct simplex approach, tJie 
Staircase algorithm tends to become more 
efficient with increasing p^coblem .size. 
^However, the threshold problem size dif- 
fers considerably for different classes of 
staircase problems. ' The r6sults presented 
here, though favoring decomposition in 
every caser .simply substantiate that obser- 
vation. They should not be interpreted as 
measures of the relative performance of - 
the two approaches in terms of absolute 
problem size. . 
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SCSOi 
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3 


3 


4, 
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ROWS 


78 


78 
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O.ss" 


4. . 

■ 491 
1633 
. . 4460 
. 0-56 ^ 


4 

^ 491 
1649 
4476 
0, 55 


4 

491 ' 
1660 
4520 
0.55 


2 


. CPU SSCOSDS ' 


30.3 


26.7 


30.3 


40.1 - ' 


■ ^ 


CPU SSCOSDS 


20.8 


19.2 


• 18-5 

'. 


^ 19,4 



Table 3. Starls^ica of the Dynamic Snergy Kodei Problems''. 
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STATISTICS^ ^^^J 


■ SCTAPl 


SCXAP2'.' 


SCTAP3 




PZRJCOS 
*R0WS 

COUIKZIS 
.' SOCTZSROS 

DESISITY . 


10 
311- 
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3683 
"l.SO 


■ ■ 10 
■1101 
2^81 
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0.44 
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Table 4. StatiAtic^ o£ • the Dynamic Tr&££ic 
Assignraent Problems. 
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' ABSTRACT ' , ;. . 

■ /• '- ■ ' . ■ ■ * ■ • . ■ . 

This paper presents a. cutting plane method for 

• integer, programming. Let a^ be the optimal point 
'of the -associated linear program ^and n be the. nor-, 
mal to the objjejptive function hy^rplane .at ap-. 
On the- ha If lines of the cone- Incident -at aQ, we 
determine those points where the half lines: inter-., 
•sect the coordinate plames a:nd then projectSJthem 
onto- nV- With the projections as stepping-stones, ' 
the jobjective function hyperplcine (the parallel 
cut)' is^. pushed into' the cone step* by step. At 
eac"h step, a minimum number of. integer paints is 
generated for feasibility* and optimality tests. ^ 
The . first fea^iblre integer point 'trapped' by 
the parallel cut is the optimal point of the inte- 
ger* program* . . . - "^r. . * ■. 

..other .main, features of .this method' -are: " 
Cl>. Iri. practice, it; is not necessary . tb, compute - 
Ithe cut. (21 in 'general V the /candidates gene-l / . 
ritated at- each • step- give better value to the pbjec^-^ . 
/it ive /-function . than ' the ■ candidates generated- at the \, 
next' step. ■ (3) - Reoptiniization is -not reqtiired^ 



search [7,8,9] • However, ^ese approaches have 
some or all of the following disadvantages s\ 

1*. There exist no' criteria for the choiq^|^f 
convex .regions {in the case of convexity .cuts amd 
enumerative"^cuts) or for the choice of halflineis- 
(in the case of cut-search) . . , ' . ■ ' . * 

2.. Extra computational .WQtk is required to 
generate the cuts. The aiaourit of work depends 
mainly on the cphvex region used. 

3- A cut may be -either .very • shallow or in the 
wrong 'inclination', i;^- too mauiy. bad candidate's 
may be generated; \ "[ . ■ 

' ' . . « . "i • '. ' . 

4. Repptimization' is" .reqiiired. .'.-^ ■ . . . V' 

.5,.- Usually, even. after a feasible: ..inteigajs' 
point has r- been 'trapped' by' a cut, . sear.cff^^^^.a '■ . 
better^ solution still- haa to.'be. .continue;^.,. 

. In. contrast, 'our -parallel-cut algorithm has 
the following main features: . . • " * 



Keywords : .Integer programming , cutting plane , ' - 
cone, normal, projections^ . 



i; No choice of convex regions or edges is 
required- A parallel cut is generated step by step 
in a definite manner- ' 



1. INTRODUCTION - « ' " 

This 'paper presents a cutting plane method 
*^for integer programming. The cut is always 
parallel to the hyperplane of . the objective func- 
tion (called a pcirallel cut) . Let aQ be the opti- 
mal point of the associated linear program and n 
be the normal to fihe objective function hyperpleme 
atap. On the half lines of the cone incident at 
ao# we determine those points where the half lines' 
intersect the coordinate pXanes, The intersection ' 
points are projected onto n.- These projections 
are their^used ras stepping-stones for moving the 
parallel cut into the cone step- by step. At each : 
step, -the intersections ;on. .the half lines are used 
to igenerate a set.of integer points foY feasi'bility 
and optimality tests- The process terminates as 
soon as a feasible integer point is 'trapped- ' " b y 
the parallel cut* / 

. Our parallel- cut method is closely related to 
three other approaches .developed in the last few 
^ears, namely, the methods of ^>convexity cuts [1,2,, 
Z',B,1X\12}; enumerative. cuts [4,S,6], and cut- j. 



. " 2-. 'A -parallel cut is merely a conceptual cutl 
In practice, no computation is involved in its gen- 
eration..- ^ ■ ' ■ 

3. The direction of a parallel cut is steepest 
■ and hence may ^be regarded as the proper 'inclina-' 
tion'- . Together with stepsize control, a paurallel. 
cut generates, at each step, a. minimum number of 
gbbd candidates, 

4- Repptimization is not required, 

■ 5. Once , a feasible, integer point is- 'trappe'd* 
bya^cutv it is optimal and.^the' process 'terminates 

Pcirallel cu-ts .are also used, conceptually, in 
Hillier's bound-and-scan algorithm ilOl , but in a 
different manner* ' 

■ ■ ;• 

2- GEOMETRIC INTERPRETATION OF' THE PARALLEL-CUT , " 
METHOD ; ■ - I ; 

. -.In -.this ^ctioA, we Illustrate by -an example " 



tihe geonietrical. motivation of cur parallel- cut 
inethod, For expository purposes / we shall use the 
following- clef initions* ^ . 

Defi^i^ion * {0-plane, 0(x*) -plane) An 6-plane is 
a hyperplane of the form cx=ci^ where, cx is the 
objective function and ct is a^'constaht, ?n parti- 
cular, ^ OC?c*) -plane has the f oisa cx=C3c* ^ where 
oc* ►is a fixed> point on the plane* 

. '» ' » '*■ . ■ 

' • In Figure 1/ land. 02 are two half lines 
incident at Sq and n is the (negative) normal. to 
the O -plane at . We observe that the coor- 
dinate planes through an integer point inside the 
cone^ntersects at least, one of the halfliiies; For 
exanple; the coordinate planes through I ^ intersect 
and at and-H^ respectively. Hence, one of 
'the coordinates of K^-, (and H^) - must be. integral. 

Definition (H-point) An H-pcint is a point on a 
half line incident at such that- at least one of ^ 
• its .-.coordinates has integral value. 

Starting from a^/. let us move- the O-plane .in 
the- direction of n- As it moves forward, it will . 
•trap' infinitely ; many integer points, ' some lying ■ 
outside the cone (e.g. X^),,^ some lying inside^ the . 

-cone but violating certain honbinding constraints . 

:(e,g. I2)'- first --feasible integer-jltoint ■ ■ ■. 



{here 1 3) 'trapped' is obviously the optimal point . 
Note that if these integer points* are projected' 
perpendicularly onto n,; the image of the optima r . 
integer point has -the shortest distance from aQ ; 
among all <f feasible integer points. • , 

Using the projections Pq rP^ rPj' '^3 ^ p^.and pj. 
as stepping-stones. Table 1 shows the H-points^ . 
integer coordinates' 2ind integer points . newly ^ene- 
. rated at each step: Note that, at ^y,- 1 3 becomes . ; 
optimal, because it is the first vteasible integer- 
point ' trapped**. ' • 1 • • • 



Pi 


H-points • 


Po 


none' 


Pi " 


. hl ■ 


P2 


Hi ■ 


P3 ■ 


Hi '-. 











integer ; 
coordinates 



integer poiht§ 



none 
..X2=l 



none ■ ■ ■ •. . .• " ^ 

none ' 
none ^ * 
1 2' {infeasible) ,'■ 
{curr- .optimal) 
(rejected, worse 



than 
none 



I5) 



Table 1 



'information .newly generated at - each step; in Fig* 




on-binding constraint' 



Figure 1 Illustration of the parallel-cut method. 
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3.' PROBLEM .EORMOIATION AND MATHEMATICAL 

PI?ELIMINARIES^ 

■. ' . * ' • - - • 

^ ' . ■• ' ■ • ■ . . ■ ^ ■ 

dbnsider the pure integer linear progr an 



basics structural variables 
basic slack variables 



{maximize." ••• cxTj^ ■ ■ ^ 

subject to Ax<b ^ • - 
* *Xi>0; integer, icN, 



(3.1) 



where N={1;2,. . • ,n}, c; x are n-vectors , A • is an • 
»<n matrix and b is an m-vector. Throu^out this^- . 
.paper, we refer to'x^, icN/ as the structural vari- 
abiles. 

. , ■■ ■ . ■ ' " ■ - <i 

The associated linear program Q' is obtained . 
from .tiie integer lineaj;' program by dropping the 
■ integrality re<5uirejnent. Introducing m slack yari- 
sibles s^colCs^rS^r.-.^s^), Q' can be written in the 
standard form • , . 



I maxxmxze 
i2**r \ subject .to 



Xq=CX 

Ax^-s»b 



(3.2f' 



Xi>0, ^icN, Sj>0, j€M,, 



where- .M^{lr2,. . . rm}- * - - . 

Suppose Q". has an optimal soluticn. Let a ^ 
and J be the index sets of the basic structural, 
and basic slack vari£ibles respectively and .y^ j . be 
the coefficients of the optimal .simplex tableau J 
'Then the cd^jective function value and each q>f the ■ 
. basic structural variables czm be expressed^' in - , 
^^erms of the nonbasic (structural and slack) vairi-. 



ables t-; as 



y. .t. r iCB- 



(3.3) 



. Let. us" repletce every y^j in ,(3.3). by another,^, 
n^^tion a£j,- for icB and j£lO}uN- Attaching, for 

f&x^ti nqnbasic structural variable x^; the trivial*' " 
relation ^tt^ng x^ equal, to itself, i.e. a re la- ^ 
tion of the form (3.3) in which' a£Q«0; a - and 
a£.=0 for i*j, i€N-B,.and rewriting^ in ve'ctor 
notations we obtain the following lineeir program 
over. a cone ' ■■ • _ - . i . 



maximize 
Cs^feubject to 



x^ 'integer, ieN. 



The fundamental relation between the problems 
C and Q is .2is followsr Let t* be an optimal solu^ 
tion to, C and x*=a„-r . ^,a.t*. Then x* is an ppti- 



mal solution to Q if and only if .x* a^so satisfies; 
those constredlnts of Q which are not "binding at j 



by 



The j*^ halfline'of the cone (3.4) is defined 



5j-ao-a.t., 



3 



(3.5) 



The quantities • a^ , a. and caj, .^^N, will be used 
in Section 4. The next lemma shows that they can 
be obtained from the optimal simplex tableau of 
the Ainear program Q". " 

Lemma 1 Suppose the following m><(n+li matrix 



is obtained from'^ the optimal simplex tableau of g**', 
where and Y3 correspond to the "columns of :the 
• current optimal -point^^uid the nonbasic 'slack vari- 
ables, whereas emd correspond to _the columns 
jof the nohbaisic^structural vauriables. Then the ; 
column vectors of (3*4) ocre. .the columns of the 
following n>fCh+l7 matrix , 



basic structural variables 
nonbasic structural vjuriables, 



wheEB l is* an identity matrix. ^1 

Fttrthermore, suppose yQj, j.^N, au:e. the rela- 
tive^cost factors obtained from the optimal . simplex 
tableau. Then. ^ 



(3.6). 



Proof. The first part of the lemma follows from 
the definitipn of . a^ . To derive (3.6) ^ we see that, 
for every t., j£N, "y'^.ES.-S. _0y. .-S.^.cy. 

"Z, s c .a . .-2 . _c . a . .«-ca . , where 6 . is. equal to • 

i€B r X2 ■ ieB r . 3 . . . . - . 

the "cost associated with t. if tj is a structural 
vauriable,^ and is equal to 0 if ^ 
■ able , "and- Bs^-B: 



tj is a slack vari- 
: (Q.E.D.) 



.4. THE PARALLEL-CUT ALGORITHM- , 

In this section, we develop the parallel-cut " 
• algorithm first under the assumption ^Sgiat .the opti?- 
.mal solution of the assdciatedz-linear* program Q * 

■ is unique and non-degenerate. - Non- uniqueness and 
degeneracy will be discuss^ in Section 4«6i ■ - 

X . ■ ■ ■ . . ■ 

■ The parauLlel-cut algorithm includes jfhe, ■ 
following . operations ; .^^ ■ y- • 

1. Generate H-points on the ha:lf lines. , 

2. Project the .H-points ^.onto the normatl n. 

3 * Us ing the .;pro j ections. as stepping-stones , 
generate a set of candidates at each step fdr 
- feasibility and optimality tests. ' 

-*.5he/de tails of these operations are described 
in the next' four subsections. i*- 

4 .1 GENERATING H-POXOTS ON THE HALFLINES 

The use of H-points wzis first developed by 
Glover [7#S] . ■ However, he used them for generating 
cuts directly; whereas we use them for generating 
projections and candidates. L^ us first restate/ - 
without proof one of his lemmas*, using our nota-/ 
.£ions.' ■- ■■ ■ ' r*""::^^'* ,■• " . ' ^' 

- Lemma 2 



(First cut-search lemma of Glover) 



Assume x' is contained\.in the truncated cone, 
of points satisfying both (3.4) and ; ' 



(4.1> 



where t*>0 for atll jcN. Then every hyperplane 
L(x-x")— 0 through x' (for L a non-zero.. row vector) 
intersects at least one . of the edges "of the trun- 



cated cone incident: at 



( i . e-. - the . Tine -segments 

In particular,, let x* be ah integer point in- 
si^ tl\e cone (3.4) , ..t^^^^fo^ every jcN, and ■ . 
L= (61^0,..., 0^1/. 0) , where.l is -in the i" com- 
ponent^. i« 1.2, ..n-. 'Then, Lemma 2 .in?) lies that ^ 
every coordinate plane through an integer point in 
the; cone intersects at least one of its half lines;. 
The H-points generated on the halfline are^ 



■3 



u ^— ■ - V 
£ .=a^-a.t. , 



(4^2) 



where .the increasing sequence of parametric values 
of t." are defined as follows: 

t^=min{t . rt.>0 and a . „-a. .t. is a nonnegative 
j. • D 3"" ^0 .... 

integer for some i} 

tj"*'^^n{tjltj>tj and a^Q-aT^^j is a.non- 
• negative integer for some i}- 

rf there is no value of t j that can be gene-* 

rated in this way, the tj or t^ is defined |o 
equal to . Niimericaliy , the values of t^ Satis- 
fying the above definition may be obtained as 
follows: - 



t°«0, t^'*'^-t^+inin{<).}, y=0,l,2,..., 

•^j D 3 i^N • 



(4-3) 



where 



rci . . . t^) -<i,o-a, jt^)/a . . if a . . <0 
((i.^_i^^ti;)-{i,n-i;,^tlf^)/a,^ if a. >0 



iO iXD 



2.2 . ID 
— otherwise. 



- <z> denotes- the " smallest integer greater " 
than z, • - - ■ 
■ and , {2}- denotes the largest intege^smaller than 

\ : z: ...... ■ ■ ' 

'■ 4.2 ; 'PKOJECTING .THE ^-POINTSV ONTO THE NORMAL' 

■ -Let n be the (negative) ' normal .to the O(aQ)- 
. plane at Sq. The distance- -of any point x-=from the 
0(aQ)-planei is g"iven by ■ . ^ , 



.d(3e)=c(aj^-x)/|cl/ . 



(4.4) 



where. B • I denotes' the Euclidean norm. In parti- 
cular, the distance between Iq and the projection 
of an H-point (4.2) oh n is given by (see Lemma 1) . 



.. d {^^) »(ca^ffc B) t^= (-Yq j/Bcl ) t^ , jcN, 



(4.5) 



-Note ' that the quantities ^yQ.<0 can be obtaijied from 
'the optimal 'siB$>l ex tableau^of the f^oblem Q". If; 

.Sq is' the unique optimal point, we have yQj<0 ^or 

every jcN. 

Let . US" denote these projections by, an • 
increasing sequence , ' j ' 



# • • • »P]^' ' 



C4.6) 



where P^^O, P-'^P-^.j- When there is.no confusion, ; 
o is used to ^denote both a projection image and 



, its distance from ao ialong'n. . 

.4.3 . SETS OF^ CANDIDATE INTEGER POINTS 

• " ' ' - . • . #*. . 

In this subsection, we. describe how ■ (4.6) • is^ 
.used to, generate integer points for feasibility 
and optiaaiity. tests. Every corresponds - to 

.one or several H-points, each of which in turn 
corresponds to bne or ^several integer values of 
the coordinates x^^'s. Hence, for each P^/^ dis- 
crete sets of values of the coordinate Xj^ 



X. Ck)=»{x[,xJ,. 



. ,x.}, ieN 

1 



C4.7) 



■. ■ - w ■ ■ . . . , ■ 

can be uniquely determined. Cv depends on i and 
k.) Candidate integer points are then obtained by 
forming all the possible combinations as follows: 

,X2 r-^ . . /X^ X^) , X .€X^<k) , icN. 

The following lemma shows that, each of the 
. discrete sets X^W consists of consecutive inte- 
gersy without gaps. 



The distrrete set X^^ ( 
&rm 



X. (k)E{x. \ Z: (k)Sx.^u. Ck) 




can be expressed 



integer } , 



Suppose there is a 
i.e^ • 



where Z. Ck) and u. (k) are non-ne^tive integer 
bounds.' V 

-■proof. Without loss of generality, we may assume 
that the" elements of Xi(k)^ satisfy. 

r '" ' q^ 'Q+1^' ■ ' - ' ■ 

osxt <xr< . . . <x?<x? < . . . <x . . 

^' ^ ^ ^ ■. q^ q+1 

crap of size g (>2) between, x? and x 

x^''"^=x^+g. Let X? and- X? -correspond to two H- 

V-i i i . ^ 

points on .the half lines 'Cq and>Cg^ respectively- ■ • 
{^e possibility that a=S is not excluded) , .i.e.' . 
there exist t*>0 and 5"*>0 such that \ : . 

:?'*:^=i:.,-5.„f, (4.8) 

s We distinguish between two cases:" ^ 
. m ■x^'*^"^>a^Q+l; and (2) x^'*'|^<a^Q+l. .For' the case 
. (1) , U.S) implies^ that -aj_gt^>l.. . .Define: 
#^S^^/^iB ^tSo-^iB^- '"^^ follows easily 



^i"^ip7"ia^a' 



that t^>tg>0 and x7 



>x . =^x . -l>x? - 



g=x.. 



This 
^ _ b 



means, that, on C^,' wie can find- an H-point a^-a^tg 

(in front of the H-point a^-ig^t^) which generates 

o Q q+1 

\the integer x. between x? and ' 
.^tion.*. Simi'larly for the case (2) , ye can find on 



XT 



a contradic- 

an find or 

an H-point which generates an integer between 



^ q+1 
and X? 
i i 



(Q.E.D.) 



As a result of Lemma 3, it is computationally 
much easier to generate candidates.*' Instead of 
(4. .7) , we simply keep track of a pair of bounds 

I (k) and u * (k) for every, x. , ieN, and every p^, 

i . .3. J- • •• I 

k=0,l,2, using the following stipulations 

r 1 ■ 

U)-.ii^c-i)HSiol' ^i^^^^'i^ioJ' ^ ^"^-^^ 

(i'i) Let S^Ck) be the set of integral 
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values associated with those H-points 
which are projected onto Pj^. Then 

adn {Z^(k-l) ,x^.} if S^ CJc) is not eji^ty. 



(4.10) 

if -.5^.(k) is emplty, " 



c . {u ,x. } if S, (k) is not en^Jty 

(4.11) 

;u: (k-1) 



if S^(k) is en^ty; 



' vrtiere [zj (or f ). is the. largest (or 

smallest) .integer *.whi^ is smaller (or /. 
■ 9real;er) than or equal tp z* . Note that 
. thfre exists r for each k, at leaist one i 
... such that- either {W<t. (k-1) or 

u^{k) >u^ (k-1) . ^ ^ 

Hence, "ttie set of candidates generated by ^ ' 
^Pq 'Pj f • 'Pic J' ®^ ^® fora . 



I (k)--{xi Z . (k)'Sx. Su-. (k) , integer^ 

: . , ^ icN}- 

4.4 . FEASIBILITY- AND OPTIMALITY TESTS 



(4.12) 



At the projection p. , our subproblem is 



maxiolxze 



»cx 



U ' 

subject to Ax<b , xcl (kj . 



(4.13) 



A direct or algorithmic sear A may be 
required to determine whether (4.13) is In feasible 
or hats an optimal solution. - If infeasible, we 
proceed to the next ^projection E^^- Otherwise, ■ 
•(4.13) has an optimal solution. ^one of the main ^ 
features o£ our parallel-cut algorithm is pro- 
. vided ■ by '. the kext. result. ■ 

"Thebrem 4 Suppose x(k) is an (optimal "poiht of . 
(4.13) . li :d(x(k) )<pj^, then x(k) is also an 
optimal point of the integer lineair program Q. 

Proof . We shall prove this theorem by showing the 
contradiction that, i£ x' is a feasible point of Q 
satisfying cx(k)^cx'. Then x'cl(k). 

' . consider the half space associated with th^ 
0(Pt-^-plane 

■• /:". ■ - ■ • • ; ■■ - V : 

Where t|-Pj^BcH/(-yQj)>0, j€N, (See {4ls) and. 
.fl].) Suppose there exists a feasible ^^int 
x*=aQ-X^^^ajt^ of Q such that. cx(k}<cx' .V*hen, -by 

(4.4) ; we have d'(x')-c(ag-x')/lcl« > 
c(raQ-x(k)l + [x(k)-xM)/Dci<d(x(k))<Pj^. and-t^ ; 

t^5d(x') Pc|/C-yQ^) <t:*^ jcN..' This implies that x» . 

sa'tisf ies ,(4.14) V By^ Lemma 2, every coordinate 
glane through x* intersects a halfline C . at • 
a^j-a^t^, say, for some t?<t*. By definition of . 

I(k), we have x'£l(k>. (Q.E.D*) 

4.5 IJETAILED DESCRIPTION ^^F THE PARALLEL-CUT ' 
. . ' JUiGORUHMv • . . • 

' '■ ' ■ . , • ■ ' 

■ A detfalled description of the parallel-cut 



..algori^thm follows: . , . 

Ste^ O Solve the aissociated linear program Q* by 
the simplex method and let a^ be its opti- 
mal point. " If ajj is integral, it is also 
an optimal point of the integer program 
Otherwise, go to Step 1. 

Step 1 ■ (Initialization) ' 

1.1 Formulate the cone problem C. (see , 
Lemma 1 of Section 3) . 

1.2 On eacdi of the half lines Ct, jcN,: gen- 
erate a sequence of H-poinis by f4.2) 
and (4.3). For each of the H-points, 
keep track if its integer coordinate (s) 

1, 3. By (4.5) , project perpendicularly the 
H-points onto the norma! n and arrange 

. * the sequence of projections in increas- 
ing order or magnitude • 

Y P-"{pQ'PirP^/.--yjp. ;. 

■\U where Pq«0, P^^P^^.^^* 

■ r.^^Let k— 1,:" ■ ' " • ' 

Step 2 (Moving the parallel cut into jthe cone) 
2.1 Let k*k+l . .■ " . " 

2*2 Solve (4.13). If (4.13) is infeasible 
or h^ an optimal point x (k) such* that 
d(x(k))>pj^^ repeat Step 2. Otherwise, 
■ ^ the optimal point *x(k), for which. 
. dCx(k))<p. , is also an optimal solu- 
tion of the integer ^ogram Q. 

In practice, the following refinements may 
be incorporated into the above formal description 
of the algorithm. ' 

Since the algorithm may terminate early in 
the process, it would be a waste of time to gen- 
erate too many H-points emd projections at the 
outset. Instead,- we may specify intervals on n 
^u^bd the half lines and generate .the H-points and 
projections within t^xese. intervals, one after \ 
another, only * when ^bhey are needed. . Supposfe £ is 
the si^e of the interval on then , the corres- 



■ponding intez 



^al size for tj is 



-.Also, step 2 of the algorithm may be replaced 
by- the following niore .elaborate one^ 

Step 2 ■ (Moving the parallel cut into ttie cone) ' 
. 2.'1» Find: the smallest k, say "k^. such 
.. that- the set {x|Ax<b; X€l (k ).} has 
> at least one feasible points (If 
such k does not exist, the integer 
^ program' Q does not have a feasible 
solution.) 

' . 2.2* Solve (4.13) Witt )c«k^. Let x(k} be 
I ^ the optimal point. 

• 2.3' If d(x(k))<pj^, then x(k) is also an^ . 
optimal point of Q. Otherwise, go 
to^Step"^ 2.4**-. - ■ ' ' 

; 2.4* -Set k^k+1-. Test whethqr the set 

T(k)H{x|Ax<b, -X£l(k)-I(k-1)," 
■.•cx>cx(k-l) } 

.is feasible. If, not, define * 
* x(k)^(k-l) and go to Step 2.2* . 
Otherwise, go to Step 2.5*. 
. 2.5' Solve the problem' max{cx I xcT:(k) } by 
. aiiy direct or algorithmic searching 
^ . .method. Let. x(k) be the optimal 



point. Go to Step 2.3*- ■ 

■,, r.* : . ■ - ^ ■ . ; ■■• 

.4*6. DEGENERACY, NON-UNIQUENESS AND PINITENESS 

in this sojbsection, we show that the parallel^ 
cut algorithm worJcs without any modification' when 
the. optimal, solution of the associated linear pro- 
gram 'Q* is degenerate. We *also discuss the rela- 
tion between the finiteness of the. algorithm and 
the urtiqueness of this optimal solution. 

Suppose (a^j,s) is the optimal point of Q": 
Degeneracy occurs when some of the basic compon- 
ents of Sq or s have zero value- Define the index 
sets • ., • '\ 

B°»{i!a^jj basic and a^^=0}r 

M^»{i|i, basic and s,=0}. 

Geometrically, in the a-space of the structural vari- 
ables ^ degeneracy implies that the. hyperplanes * 
associated with the halfspaces Z; „a. .x.<b., i€M 

■ o ■ ^ ^ ' 

or x.>0,' ieB also pass through a^. In the same 

spacer let us define ' 

"x-{xlz:. „a/.x.<b., ieM; x.>0, :j€N} 
and ' ^ " ■ ^ ; * 

Z X* = {:<\Z^^^a^^X^<b^, i£M-M°; Xj>0, jcN-B°} . 

Thus, .X' is obtained from X by dropping the con- ' 
strain ts associated with those variatbles (struc- 
tural or slack) which are basic but have value 0. 
Obviously," Xcx* . Balas [1] proves the following 
lemma. . _ . 

Lemma 5 -a^ is the vertex of X* and.cx^ca. is a 
supporting hyperplame. for X'. X" has n distinct 
edges adjacent- to aQ,'and each half line (3.5) con — 
tains exactly one such* edge. ' . - . 

Interpreted geometrically (see Fioure 2) . 
this lemma .implies that, in case of degeneracy, 
some of .-the .halflines (3.5) may not contain any - 
edge of the^cone (incident at a^) of the feasible 
region X of the problem Q*. However, each half- 
line (3.5) contains an edge- of Mother polytope 




Figure 2 degeneracy at a^ 



In connection with our parallel-cut algorithm", 
* it is obvious that the half lines (3.5) can also be- 
used to- generage H-points", which in turn are used . 
to generate integer points to be tested -for f easi- 
bility with' respect ~Tro X. In other words, the 
algorithm works without any modification. 

There is a close relation between the tinique- ■ 
ness of. the optimal solution of the associated 
linear program -and "the finiteness of the parallel 
cut algorithm. - , . < *\ 

if the optimal point of ,Q' is unique, the 
following argimient shows -that the algorithm is 
finite. Let x' be ah arbitrary feasible integer 
point (assumed exiG ting) , of Q. then, the 0(x')-. 

. ' plane intersects every: half line at a finite point ■ 
and d(x*) is an upper bound on the projections to 
be generated. Thus, .the numbers of distinct 
H-points and projections are finite. Hence, the 
number of'stfeps and the number of candidates to be 

^ tested at each step are both ■ finite- ' ^ 

If some of the relative-cost factors y^. are 
zero, the optimal point- of Q* is not 'Unique - 
Define the index .set D5{j|j£N, yQ.=0} and .the poiy^ 
tope . • 

Geometrically, non-uniqueness implies that X*V ' lies 
on the 0(aQ).-plane (see .Figure 3). Hence, alJ^-the- 
H-points and in.teger points lying in X" are projec- 
ted onto , and the discrete- S'et -I (0) as defined 
in (4-12) may not be empty. We distinguish between 
the following two cases: > . - 

(i) - X is bounded- _ ■ , - 

For such a case, the number of H-points gene- 
rated on each' of the ■ half lines C, 'jcD is finite- 
■ By similar argTament as in the uniqueness case,, we 
can show the parallel-cut algorithm is finite if 
boundedness is incorporated into the definition 

.• :(4.3) : . . ■ ; '* • • ■ 

Cii) X is. unbounded^ 

For suci^ a case , it .may be necessary to gene- 
rate' infinitely many H-points on, some of the^ C. ," 
^€D. If X" contains an integer point, Q becomes 
a problem of locating any such integer point. How- 
ever, . it may happen that, theoretically at leastv' 
^ X" does not contain an^r integer- point but there • 
exist feasible integer, points /of Q which are arbi- 
trarily close to X".' (See the remark below.) This 

^mplies that, at the initial projection p^ of the 
algorithm, "search for the. best feasible integer" 

"ioint over the infinite, set I. CO) never termijiates. 

" 3n practice , such situation may be . remedied by 

-^introducing , an artificial constraint, (the dotted 
line of Figure* 3). Then an approximate solution ■ 
is , sought ' over a bovmded portion of the unbounded 
feasible re^ipn. • ... 

Remark: . It seems to the author- that, theoreticalil^y 
speaking, this situation -gives rise to troXible for^ 
most of the existing integer programming .methods.'' 

' • J : 

" -J _ ■ ■ * ■ ' . , ■ • " 

■ 'IS 




Figure 3 -Non- uniqueness of - optimal point 



.Application of. the underlying ideas of the 
approach presented. in this paper can obviously be' 
extended to mixed, integer programming^ and probably 
.to- nonlinear integer programming 'with 'linear con-^ 
straints. ' ; 
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Abstract 



A monomial programming^ problem is on^ 
of' minimizing, a polynomial in several 
veiriables subjec.t to monomial constraints. 
A log transformation, changes, it into a 
problem . with non-linear objective, and 
linear.. constraints which,- under Certain 
conditions cam be solved by Zangwill's ^ 
convex simplex method. We show a direct 
jnethod, based on our previous work; of 
solving the problem using a simplex-like 
tableau. . - 



1. 



INTRODUCTION 



In a recent series of papers [3, 4, , • 
5, 6], we have presented simp lex- like 
algorithms to several Special kinds of 
problems arising in modular design, 
geometric ^programming, and elsewhere- - Xn.' 
the present paper we shall specialize the 
general algorithm of [4] to the case of a 
programming problem in which the 
constraints involve monomial expressions. 
For this case a true simplex algorithm -is 
possible in' the' sense th^ any of the 



This report was prepared as part .of the 
activities -of - the Management .Sciences 

^ Research^ Gj;oup^ Carnegie-Mellon Univer- 
sity, . under. Contract #N000l4-67-A-0314- 
0007 NR 047-048 with the .U.S, Office of 
Naval Research. Reproduction in whole 
or in. part is permitted .for any purpose 

"of the D.S. . Govemment*' - - ■ 

Management ^Sciences Research ^(eport #333r 
Management Sciences Research Group, 
Gi^aduate School of Industrial Adminis-' 
trat.ioh, Carnegie-Mellon University, 
Pittsburgh, Pennsylvania 15213. * 



standard' linear programming routines ' can ^ 
rather easily be modified to solve 'monomial 
cons.trained problems,- Also all of tbe. 
techniques for handling large scale" linear 
programming problems, such as the revised. . 
simplex methods decomposition, lexico.- 
graphic ordering, etc.,. are immediately 
transferable to the new> algorithm. >It , - 
therefore follows thatf mpnomially 
constrained problems in hundreds of 
variables can be solved. ■ ■ 

In Sections 2-7 we present .a . 
description of. the new method. Computa- 
tional results""are presented in Section 7.' 
A monomial model ^nceming.. excess ' ^ ^- • 
inventories is presented and solved^in 
Section 8, ' ' 

,NOTATION . " ' I . - 

Because we wiXl be working with' 
monOmiaUx expressions -and their derivatives 
in several variables we introduce some . - 
special vector ancj matrix notationto make 
it easy to do so. We define. the, O 
product of an n-component row vector Jtx -and 
an n-comiponent ; column vector x as: 



(1) h O x = (h. 



' . -^1.^2 

We extend' thi^ defihi^iion in the obvious 

way^ to the .'O product of an -mxn matrix H 

and' an n-component-, column vector^^x as: 




(2) H 



O X = 





where h\ is the ith row of H, 



As a 

numerical example observe thatr ^ 
12} { 0 i 31 VT^.." I I . \ ' - (; (4/3)^ 



tfsing the (T) product and t^he 
ordinary matrix product, we can easily ' 
.write- a polynomial in several variaibles; 
let d "be an in-component row vec'tor, H *an 
' mxn matrix^ and x an n-component column, 
vector; then: 



(4) 



d(H Ox) ='cij^x^ 



11 In , 

• ; - X 
n 



mn 



* m X r , . n 

is . a polynomial in the variadbles, / . ", 

- We recall next the definition of the 
Schur pr.oduct o-f vect6rs. Let a and b be 
two m-component row vectors; then the 
Schur product is defined asr 



(5) a*b = {a- 



m 1 

a b ) ; 
m m 



.••'V = 



using this we can write the. derivative of 

,^ 'polynomial. d(H 0 x) „ Let h^^-^ be. the 

jth column of H and let h-^-^' ' be its 

transpose; let H^^^ be the matrix. H with 1 
subtracted from each. entry in the jth 



= nx 



column. Since tne formula 
works even when n=0, it follows that: 



dx^ 
dx 



n-1 



(d*h 



IS) ^ 5^ (d(HO x)) = 

From this it further follows that; ^ ^' 
(7) 



0x). 



X ^ 



(dCHOx)) =' 



(d*h 



J- 



y (H Ox) . 



f = 'Sx^x^x^ +■ 7x^X2 



As an example, 
can be expressed in the form of equation 
(4) ' with d =,* . (6 ^7} and H = 0 



.£f 
3x. 



2 3 
^2^3 



3 S 
^^1^2 



af 



4 5" 



,12 3 



can then be expressed as in 



equations (6) and (7) with h ' =(1,4) 
andH.(l> " ^ 

3. '- PROBLEM STATEMENT ' 



= /0 2 3\ 
\3 . 5- 0>.- 



By- a monomially constrained problem 
we jshali me^ a. problem of- the " form ^ ^' 
Minimize.' d.(C.0x). = g' . 

-(.8) Subject to aOx >^ b . . 

where d 'is. Ixk, c is kXn, A is mxh,^ x is 
nxl, and b is mxl. We assume b>0- * . 



.Note .that by making the' tr ems forma- " ' 
.tion: 

(9) X. = e ^ • ■ . 

■ and taking logarithms we could change (ff ) 
into .the problem (using the obvious 
definitions): ■ . 

'Min'imize^ ■ • d"(C'O e^) *• 

(10) ; . ■ v 

. Subject to Ay tnh 
Problem .(1) tias linear constraints^ but y 
is. not constrained to be nonnegative- 
Followihg Charnes and Kirby - [1] we then 
say that (8) is trans formably^ linear, 

. The. following facts are" well known. 

-If d>0 then d(cOe^) is a stiri'ctly convex 
' fxinction of y^^, - - - ^y^. If d>0 then the 

same fumrtibn is -a convex (but nbt' 
necesisarily strictly convex) function of 
these variables- * 



■ If d(C0e^) is a convex function of 
^ y we could use Zangwill's convex simplex 
method [9] to solve problem (10) and thus * 
(8). However in this paper we shall 
specialize our previous work [4] on * 
- simplex-like methods for solving -nonlinear 
problems with nonlinear' constraints to 
provide a simplex metho^ that is somewhat 
more general^ than ^'ne- c6n-o:< simplex 
method for solvinc problem ^5 directly - 

Let X = ^^1' ' ' ' ' ^rri^ vector 
of Lagrange mukLtipliers for the -.constraints 
■■ ojE (8)- Ther^he Lagranqian of' (8) is: 

(11) d(c(Ss^) - A(A(3)x-b) . • ■ • ^ 
where A O ^ = b ^now is considered to 

. include the constraints x>0. Taking the '"^ 
partial derivative of-^L wrth -respecl: to' 
Xj, using 16)., and setting, the result equal 
to zero yields: 

(12) |i-= "^d*c^^^')(C^^^Ox) v. 

^ .'^ . ; - (X*a ) .(A^3) Q^y ^ 0 _ 
.Multiplying (12) jDy x. and use (7) to. 
obtain: ^ 

-..(13) (X-^a*^^.^'l(AOx) = (d*c^=3)')(cOx)/ 
We now use the well-known Kuhn-Tucker 
complementary slackness condition: 
. (14). ^j_ Ca^^,^ O^) = ^^i^i i 

where a^^^^ is the ith row of A and define 

(15) 0^ = A^b^ ^for all i. 

Substituting '(14) and , (15) ' into (13) gives: 

(16) ,,pa^^? = (d*c.^^^')(C0x) for all j. ' 
We shall call- the dual varaables and 

, (1^6) the dual equations , 

As in linear", programming duals 
associated with constraints^^ 0% the type " . 
•x>_t) must be. zero for basic variables, so 
that at the optimum (16) must be satisfied 
for the original m constraints. If we let . 
the right hand side of (16) be^q^ for the *^ 



29 



51 



basic variables then in matrix form pB = q 

— i . 

and p = qB' where B as the basis. , *Like- 
wise ignoring the nohnegativity con- 
straints (16) becomes: ^ 

■ . .{d*cf^^ ') (cO X) - pa^^' < 0- for 
nonba'sic variables:^' - 

The left Jhcmd side of. this "inequality. ^ / 
corresponds "to the " reduced cost "^' r^,, 

values in linear progreunming,- or ,1 • * 
explicitly:.. 



tlSb) 



r..= (d*cf=i^" 

all j nonbasic, 
.-1 ;(j) 



) CC 0x) -qB"^a^^^ far 



where B is merely the jth column of 

.the present teibleau. 

4 . ■ RELATIONSHIP ;T0 LINEAR PROGRAMMING . ^ 

Note, the similarity be.tween the^ 
ordinary, linear progrcumning dual, equations 
and (1.6) ./y. If ' the ith: constraint in (8) is 
not tight p^^ = 0 and if it is tight p V >_ 0. 

Hence we . can use the p^*s^or equivalently 



X' =-cr-i' in the same way as they 
Jin ordinary linear - programming- 



e -used 



If we assume that one pf -the monomial 
constraints -in (8) ' is tight and use. it' to 
eliminate a variable/' say - x - > from the 
rest- of the 'equations then "^it is not hard 
to see tt^.t .ihis tremsformatioii will make 
linear changes in the exponents of the., 
other monomials and log linear change in 
the right' hand sides - 

For inst'ance, consider* the polynomial 
equations: ■ - 1:. / 

x^ - 3 ' ' ^. ■ . . 



= 4 



Using .the notation of the previous section 
these"*can be written A x = by or in 
detached coefficient , form as: 




where -A is the 3x2 matrix on the -left and 
b is .the 3xlj vector on the right- Hif we 
now want to use the , second equation^ to 
eliminate X, .from the other two equations 

"we can multiply • by -the corresponding pivot 
matrix (see-[?]) in'-front as follows: 












2 




1 2\ 


3 




6 


3 


1 


2 1" 


9 




. 1 


.; 1 

2 


3 


1 >1, 


4 




■ 0 


.1 
2 


.4 

^ 7 



Note that we used ordinary, matrix multipli^- 
cation in the A area of the tableau, but 

' 0* multiplication in. the. b-area. All. the v 
fauniliar rules of pivoting in linear" 
programming apply . if they are appropriately 

^ modified for operation on the right hamd- 

'.>side. '. 

*; Mos^t o'f the problems encountj^red: 'in 
solving ordinary linear pfograinming pro- 
blems such as^ degeneracy (which can he 
handled by perturbation techniques or 
lexiographic orderings). can occur, and 
some^new difficulties as well- For. . . 
instance it is possible that one of* the . 
■factors of the objective, function couii 
. tend to z^fo without a basis' change being 
• needed. To heuridle, sych- problems we caa V 
employ regularizatiori constraints.^; the 
form (C O X") . > > b where kv is a small 

number. .Most of the other difficulties 
can be -handled by""' siiailar modifications of 
well known linear programming techniques ' 
-and- will nbt be discussed further here. 

We nex,t describe a simplex method, f. or » 
solving- problems of the " form (80 - ; To start 
"'it we need a Phase^ I procedxire* To this, . 

. end we' add. slack variables 2. and ' • 

. . ..^j^ .1;. ■ . 1 ■ . ^ . • 

./artificial variables u^ to the- cons.t^raJLnts 
of '(8) to obtain, the equivalent ptoblem: 



(17) 



Minimize d (C 0 x) 



Sxibject, to 



(A, -1,1) O 




= b- 



'Let f be a column, vector of all ones-" 
If b>^f it follows that an initial feasibl'e 
solution is x=f/ u=b, so that the initial * 

■ tableau for the simplex. Phase I start is:v 

- " - .. -x *2 ■ 

^ (18)" 1 A^, -i ■ bj u,- 



If however/ some components, of' b axe ■<1 ^ , ^ 
then 'the initial feasible* ba^is can be made, 
up of the components of a. 'for which b. 

and the -comp6nents of .2^, for:^wliich ^^£,1- 

VJe now use a Phase I objective "function . 
e O u (where e is an m-coluran vector of 
all ones) and pivot until all the u^ y • ^ 

artificial variablres have been elimin'ated 
in . complete analogy to ordinary linear ' 
•programming; ^ ^ I 

'In -the next section. we describe in' 
detail the simplex method for monomialiy 
constrained problems which is based on the- 
general method adescribed - in our paper [4]. 
The .main dif fer'isnces from the ordinary 
linear prograimmin^ simplex method are that 
•the :variables\a.re not constrained to be >_1 
and second that rfonbasic variables can 
take on values other than 1 or^-" The.- 
latter is necessary in order that we can 
find optimun .solutions ^at are not neces- 
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sarily detennmed by intersections of 
constraining hyperplanes. We use this 
method of describing the algorithm rather 
,*than the parametric programming method of 
[4] because , it seems to be simpler in the 
present context. . . 

, 5. ■ THE SIMPLEX ALGORITHM FOR MONOMIADLY 
: CONSTRAINED. PROBLEMS > 

.V.-. ^Xhe.c^escription of the algorithm is . 
. similar to that for ordinary linear pro- 
. * "gramming. All *of the variants of the . • 

latter can be ihade to the present ^ 
^ algorithm without difficulty so we do not- 

gp ^^intp dei:ail in their discussion here - 



■I 



Phase I. 



Set-up. the jJjroblem,. find 

the " quantities, . b, axid d and' sub- 
stitute them into the ini:1:ial -tableau (18)-, 

(A) Go to -the. MAIN routine using the 

objective function g = u-u-...u - rf ' the ' 

• . - 1, ^ m 

problem has a solution with value greater 

.'than 1 then the original ' problem, has no , 
solution; stop. If the problem has a 
solution with value 1 then all the 
artifipial variables are nonbasic (or can / 
easily be made nonbasic and eliminated 

■ from the tableau). ^- — 

Phase II > Set g = d (C O x) and go to 
'the MAIN program- When that program is 
complete it 'will provide an. answer to the 
originally stated problem, or to -the ' 
regularized problem if. infinite solutions 
are found. ; . 

» * ' ■• . 

MAIN program. ... 

1. Dual Solution Routine 

■ . (a) For/e'ach basic .variable v^/ - 

calTcuiate the. value of^g^ at the 
•' \ current solution. 

•■■ ■*■" ^ - ■. ■■ . . 

* (b) ■ Fpi?T^ach nonbasic v. calculate 

'di''--- , " D ■ 

^ the reduced cost^ r^ , using 

(16b) where the second term 
sumation is to be taken over the 
basic*' variables and hence the 
sum 'calculation can be done from 
the current tableau - 

2. Find Incoming Vector. 

Calculate "ithe sets 

= {Vj'|v^/is njDn basic, v^ ' 
is tight -at a" lower bound and 

r^ > 0. } : - 

= .{v-..|v. is non basic/ v. 

is 'not tight at a lower bound 
and r . ^ 0.3 * 

' '\ "If S^ U S2 = 0 the current solutionj 



6. 



Max 
keS. 



optimal.." . , ■ 

Go^to 7. Else^go to, '3. 

3» Find largest .indicator . v. 

; .^Find j" as .th'e index that maximizes- 

Max{Max Ir, 

kcS^ ^ ^.^2 

. - '-If j€iS^ or jcS^ and rj>0 go to 4.' 
If j£S2 and rj<0 go to 5- 

4. Find solution with larger v^. 

^a)^^ Find the maximum extent vt to 

which v. can be. increased while 
' . ■ ■ • 3 .• ■ . 

keeping feasibility- This csm be 
done by using the .column of the- ■ 
currex>3: tableau under - and the" 

• ■ . " . right hand side column - 

Determine outgoing ve^ctor Vj^» 

(b) Solve the auxiliary problem -.for 
vciriable v.; let v? be its ■ 
optimum value. ■ 

^ * (c) If V? go to^.e. , 

■(d) If v"/ > v9, se/fc v.=v? in the \ 

. tableau, calwlate the modified 
right hand sides and go to 1.^ 

5. ' Find s^lution^^ith smaller 

(a). Find the maximum extent v^ to " 

rfWhidh Vj- ican be decreased . while 

k-eepingr fe^asibility.^ This c^n be 
done by usincr the . column bf the . 

current tabreau under v. and the 
.1 3 ^ . 

right hand side column. 
. >^ ' " Determine the ""outgoing vector v^^. 



(b) 

■(c) 
(dO 



Solve the' auxiliary problem for 
variable "^j ^^s 

optimum valua|P 



* I f V . > V . qo to 6 . 

" . 3 - O ' . ^ 

If V. < v9, set v.=v? in* the. 

3 3 3 3 

tableau, caloi^late the modified 
right hand side arid go to 1. 



Pivot exchanger make v. basic and v, 
. 3 h 

nonbasic. Let P be thfe corresponding 

pivot .matrix (see [2] ) ; then use the 

calculation PA^°^^^; = ^^Cnew) ^he' 
A part of the .tableau and P0b^°^^^ = 
j^(new) ^ p^^^ of .the. tableau 



End of MAI^ -program. 
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• AUXILIARY problem routine for ntbn- 
basic variable v^.- . . 

■ 3 

4. ./Evaluate the objective .function g.at/ 
the- ciirrent solution but aissvroing' v. 

is a' variable* Let gfVj) be the 

\ ' : corresponding function. / 

2.' Use/a' search {or other) technique to .| 
/ . . fecnd the unconstrained miniinum value 

, ^ /Vj of g(Vj) . 

6. EXAMPLES • ^ 

" ' '■ * ■ , ■ • . ^ ■ ■ ^ . 

We Work two examples in oi^d'er to 
illustrate the method- We staTte' first the . 
cohs^aints-: ^ ;■ - *■ ' ; . ' ' 



(19) 



The feasible regioli is^hown in Figure 7. 
Adding slack and artTficial- variables' 
these become'. ■ / ■■ " . 



i20.) 



2 -1 
^1^2^ u^ .= 3 

■2 ^-i - 
^i^2^2 ^2 = ^ 



^x^x^2;^U3 



= 4 



The • constraintSL 2C^2_1 r ^2-^ Vill be imposed 
by the method in order to keep tableaus * 
small. . We now use the Phase I objective 
function u^u^u^ and do the phase :I calcula- 
tions in condensed, taibleau form (Figure l)v 

for; 



It IS easy* to show- that . 



i = 1# 2,3. / The corii^esponding dual 
varialbe calculations appea;i:--on the right 
and below the/tableau* The. pivot element, 
is circled; the new tableau with ^variable 
u^ dropped appears in Figure 2. The (J) 

operation wa;^ performed on; the right-hand 
'side. ' ^There is one positive indicator ., 
(reduced cost) in column 2.- -The second 
and third rows indicate the relationships 



3 -2 1 - ■ ' *1 • 

^2 " ^2^1 ^2 ^3 " (4/3)x^z^ z^'so that 

/whfen-^^ is decreased u^ and u^ are 



decre'ased. 



Since we don't want to qo 



■ below 1 we pivot; the new.' tableau with 
variable U2 dropped appears in Figure. 3.- 

Using- the same reasoning as before we 
. • 1 * " 

pivot on the circled in -Tiqure 3 and • ** 

. i 

^obtain the primal feasible tableau of. 

• Figure 4. The solution x^ = 3/4, vi^ = 16/9, 

2^ = 64/27, Z2 = = 1 and - ^2 ~ ^3 
is feasible for the constraints' (20) . 

Suppose now we wish to solve the 
' problem: / ^ 

4 " 



(21)- 



Minimize ^-^2 ~ 



l'^2^ 



Subject to constraints' ^19). . 
The tableaii. of Figure 4 with dual variable 
calculations for this objective function is 
„ shown in Figure '5, Olf ^e increase z^^ we 

have x^ = \^/4)'f^ and X2 = (4/3)^Z3;so 

that gCz^) = '(9/;4)^(4/3)^z^^ so that 
iiiczfeasing decreases gCz^^^ indefinitely. 
Biat the Mmit on increases of z^ is the 



^requirement x^^^l. 



Therefore we - pivot on, ' 



the circled. 1 in Figure 5- giving the ' <* • • 
tableau of Figure 6. ^ ; 

•Hence the optimum* solution is x^ = 1, ^ 

^2 ^1 ^ ^2 "^-^ ^3 ^ which can 

be seen in the graph^f Fig;are 7. The " 
value of the objective function is " 
g(x^,X2) =■ ^. . . " . 

• ■ ' - ■ ' * 

We. now solve a problem with the same 
constraints but a different objective 
fimction in order to show the solution of 
the auxiliary proble9i4 The new problem- is: 
■ --^ - ' - 



(22). 



Minimize x'x-, + 2x7^Xo 
Subject of constraints' (19)-. 
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-1 
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-1 . 


• 0 ■ 


9 




UjUjU^ » 108 




1 


■1 
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. - 0 


• -1 . 
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-108 


-108 
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"2 


=^1 


^2 


"=3 










2 

• . -3 


-1 

0 


0 


. - ■ 
0 


3 . 
1 ■ 


"2 


"l"2"3 ■ 


4/3 


-1 


1 , 


0 


-1 . 


4/3 


"3 


"l"2"3 - 


4/3 


-16/3 




-4/3 


-4/3 











Figure 2 




1 
4 

3 ' "3 



, Figure 3 



0 



^2. 


■ ^3- . 




-1 


1 






-3 








CA/3)^ . 



Figure 4 




9/4 



C4/3)- 
(A/3)^ 



-3(9/4r(4/3)^ 2(9/4)^(4/3)2 

Figure 5 




9/4 - 

(^/3)-^.(9/4)^ 



4x^-2 « 4C9Vi).^(A/3.)^, 
'4 ' ' L ■ 2 



.0 
'0 



(A/3)2(9/4)2 - 



4 

^ ^2 



'^4 ^ \4. . -4 • 
-Xj^ 2xj^ x^-Ax^ X2: 



'Figure 6 
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The tableau of Figure A with dual 
varicUale calculations for the new objec- 
tive function is shown in Figure We 
see that we shoiild try to increase as 

in. the*. previous example. From Figure 8 we 

see that = (9/4)z^'^ and x^ = (4/3)^z^ 

so that the auxiliary problem is to: 

• Minimize g{z^) = (9/4)^(4/3)^23^ + 

2(4/9) (4/3)^23. 



Differentiating g with respect to 2 



3' 



setting g» (Z3) = 0 and solving gives the 
solution.: 



;Xj^^.= 1-25 

=13.97 



x^ = 5.80 
2^ = 1, 



Z3 =1.81 



gtXj^^x^) = 23.28 

The finals dual solution: shows, ^that this; 
solution is. optimal as shown in Figure '9-. 
Figxire 7 shows the. graph of g 1x^,^2) = 

23.28 touching the boundary of the 
■feasible set at the optimum point. 

7. COMPOTATIONAL RESULTS ' 

, / Some computational ; results. are shown 
in :Tab3,es 1, -2 and 3. The/ptogram used 
was* written . in Basic cind run on a 
Burroughs' 6-700. Processing time 'included 
all central processing' .time e^dbept that* 
used for input/output, -^he probem para- 
meters were generated raifdomly.. ^ The : 
exp<^ents for the constraints were - 
intrf^gers between -minus one a^ three ■ 
whiJSe the right hand sides were integer 
numbers between one and one hundred. The 
exponents 'xh the objective function were 
random integers between zero and fa^ ' 



multipried by one half. Finally, the term 
• coefficients in the objective function 
were random integers between zero and nine. 

In the "three "tables a pivot means the . 
same as in linear programitfing i.e. , a ; • - •. 
basis change. A pass is the operation bi 
modifying the value of some variable- 
thereby changing the solution. In some 
cases^ of course, this modification will 
lead to a pivot but not always. As we 
would expect from linear prograuraning, ♦ 
pivots seem to be very fast.* Problems with 
a high proportion of total pass^es being 
pivot operations have comparatively low -r 

. .solut-ipn times. As is obvious from 
Table Ivproblems with many constraints 
relative to* variables are solved mostly 
through pivoting. This fact seems to 
result from the reduced feasible region 
and the resulting high probability of 
finding the optimum at an extreme point or, 
at^-least, the 'intersection of several con- 
straints. This can be verified again in 
Table 3. As can be seen in Table 2, more 

^complex. objective functions also seejn to 
block out extreme point solutions^ 

In presenting these computational 
results, certain considerations must be 

made concerniTig the program itself;. 

. ^ ■ . 

^ . ONE. DIMENSIONAL SEARCH PRdCEDURE. - 
'since pivots can be '-done rapidly compared 
to searching^,, the seairch procedure is not 
used unless pivoting would y^ild a worse, 
or equal solution than the. initial value'- 
(with the exception that for degenerate 
sitiiations pivoting is .always performed) . - 
^This criteria seems to reSuce the total 
"number of searches ^hich must be performed 
thereby reducing total ♦time. When • 
.searching in undertaken a -combinati-oh- " 
quadratic fit and Fibonacci search c 




-133-53. 86.39 



1 



, 9/4 . 



Figure 8 



4(9/4)^(A/3)^-2(9/4)"^(4/3)^ - Z80.67 



.<9M)'^('i/3)^+2(9/4)'\4/3)^. - 47.14 



Conscxaints 



Phase I 
VfiAablcs Pivots 



Phase II ' Processing 

Variables Pivots Passes Time (Sec.) 



■ 5' 


> 


♦ 15 


* 6 


10 


0 


. 1 


1.1 


5 . 


20 


" 5 ■ 


15 


4 


6 - 


2-5 


5* 




- 30 


5 


• 25 . 


4 


58 


48.0 


10 




25 


. 11 


15 


1 


- 1 


1.3 


10 




30 ^ 


♦14 


20 


5 


6 


3.5 


10 • 




35 


16 


25 


8- 


* . 29 


24.3 


15 




45 


21 


30 


12 


. 13 


7:5 


15 




50 


26 


35 


16 


21 


17,5 


15 




55 


25 . 


• 40 


24 


" 44 


47.4 


20 




45 


• 25 ♦ 


25 


1 


■1^ 


4.2 


20 




60 


37 


40 


6 


6 . 


10.5 


25 




55 : 


30 


30 • 




• 3 ■ 


7.4 


25 




60 


42 


35 


3 


3 


10.7 


25 




65 


47 


.40 


.6 


6 * 


13.8 


25 ■■ 




70 


54 


45 


' 15 ■" 


15 ■■ 


20.5 


25 




75 


45 : 


.■ 50 


27 


46 . 


95.5 


30 




70 


52 


40 


5 


5 ' 


16.8 


30- * 




80 


73 


50 


22 


■ 22 


37.5 


30 




85 • 


75' 


■ 55 


23 


23 


39,2 


35 




100 


68 


65 


22 


22 


51.6 



Table 1:. Time results. Objective .function 
has five terms -.no unconstrainted 
, ■. variables . ' ; 
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.inns in A - , Phase l\ ' Phase II ' Processing Time 

ObjecSve Function - . Pivots Pivots Passes 



5 
10 
15 



5 4.6 2,5 

5 .3- 5 . ■ ^-5 

5 ^ ■ 3. 35 .73-7 

Table 2: Time results- * 5 constraints 

20 variables in Phase I; 15 variat)les 
ixi Phase II - no unconstrainted 
variables. 



Unconstrained 
.. Variables 

0 
5 
0 



Constraints 

5 • 
• 5 - 

10 

' -10 • ■ 



Variables ' 
Phase ^I Phase II 



'15 
15 
30 
30 



10 
10 
20 
.2*0 



Table 3: 



Phase I 
. Pivots. . 



. Phase II 
Pivots Passes 



Time results . 
has five terms. 




Processing 
Time (Sec») 

2.5 
8.7 
3.5 

28; 3 ■ 



■ procedure is used. This-- portion of the 
program was "written to take advantage of 
the speed of quadratic prediction and the 
^robustness: of the Fibonacci search. . 

- Search is stopped only when the machine, 
tolerance (twelve significant digits) in 
the objective function is reached, i.e.,. 
the predicted points are so close that 
the objective functions are equal to with- 
in twelve significant digits. 

STOPPING HULES. Stopping rules in" 
nonlinear programmi^ig are by necessity 
extremely complex. Straight limits on 
the maximum allowable size of the 
reduced costs associated with movable 
yariablie.s /is .difficult since thfe impor- . 
tance. of. "these reduced costs can only be- / 
considered in terms of the absolute size 
of the objective function- Also, when the 
one dimensional search is used the result- 
ting minimum solution is only an approxima- 
tion which leads to nonzero reduced costs 
associated with tjrfe last variable moved. 

- (Exact solution^ in -the liners search would, 
of course,' lead to a zero. reduced cost for 
this variable). ^Typically even small 
deviations in the- linear search seem to- 
yield large nonzero . reduced costs - thus 
the choice of as fine a tolerance as . 
possible in the search portion, pf the 
program. From these .two considerations 
two stopping rules were developed both. of 
which lead to program termination. The 
first- rule is used after a variable. is ' 

.modified but no pivot has occurred- The 
reduced cost '^associated* with this 
. variable should be zero bur will never 
■actually" equal zero- Di;iring the next, 
pass the reduced costs are tested against;, 
reduced cost, say r^, of the variable. 

just modified. If no other variable 
which is a candidate ' for modification 
under section 3 of the algoi^ithm' has a" 
reduced cost whose absolute value 
exceeds 2r^ then the program stpps and 

the present solution is printed out as .1 
optimum- The secpnd criteria is iJlvo}ced^ 
after a pivot. This rule stems from .the ^ 
fact that the reduced cost is v.3q/3v.. 

We x±sh to stop when ;the objective 
fxinction is changing by extremely small- 
amounts relative to the objective 
function. In our program if no 
candidate for^ modification has a 
reduced cost whose absolute value 



-7 



^-5 



exceeds 10- .-gfx) or (10 ^.whichever is 
greater) then the ^program -stops and the 
present value is printed out as^the 
optimum. Th^ stopping rules for the . 
computational results displayed in " ^ 
Tables 1,- 2 and -3 were purposely, very^ / * 
stringent- Figure* 10 shows the converg- 
ence properties of two of the longer runs. 
-In these situations the . objective 
function levels off very quickly. These 
results? indicates that for practical 
purposes- the times shown are"^ quite high, " 



CODING EFFICIENCY. '.Use of equation 
(7) when calculating partial derivatives 
enhances- the program considerably since 
for a given solution the bulk of this 
calculation H O x stays constant across. all 
variables. It should be pointed out that 
similar efficients were not coded into the 
search routine. In particular the 
objective function is reevaluated- for each 
new point of the search procedure, even, 
though only m+1 variables of the total 
variables are being modifie^dv , The linear 
search itself seems to be t^me consuming 
which suggests that improvements bere are 
possible- Indeed.it may prove to £e' true 
that high precision in these searches is 
not necessary early in the solution . 
procedure r 
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EXAMPLE PROBLEM 



' Due to exogenous .economic factors an 
automobile manufacturer is faced with a 
large, inventory of completed new automobiles. 
The company sells three basic types of 

'cars - large, mediujff\and small. " Each size 
has, a different number of cars in 
inventory. The manufacturer also knows 
that sale| of one'.'type car has a 
■signif icaiTt effect on sales of the other 
two types. . ■\The. manufacturer wishes to 
determine a short term price structure 
which will allow him to maximize revenue 
wjjile turning a significant proportion .of 

- hi*s surplus inventory. 

Sales of each type of car are fore-^ 
'casted via the following Cobb-Douglas * 
equation: 

sales of car type i = S.V= 



P2 



'i2 



'13 



•i = 1,2,3 



where are the prices for the three cars. 

Revenue from total sales is of coiirse ■ 
^ P^. Assume -^e know the following' 
■parameters which predict s^&les: ' 





' 1 


2 




1" 








2' 








3 ' 


; .5 


.3 


-•5 



1x10^ 



8x10^ 



3x10^ 
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and 



Inventory - • 



45,000 



128,000 



i5o;ooo 



Desired Sales 



25,000 



6A,Od0 



90,000 



Maximiam. and minimum prices as set by 
management are: 



Lower 


Upper 


bound 


bound 


none 


/. none 


none 


5,000 


1,000 


3,000 



■ Management wishes to determine a'- 
.price , for each car which maximize net 
shoi?t run revenue from sales* 

The monomial model for this problem ' 
will be r 



Max. Op^S^'p^^^ ^Pl'^^^^'Pa'"^ 



.A5 .5 -A 



^ *5 .3 .5v V -.3 , 
3p^ . P3 ) X .10 ^ 

S-t. 25x10^ ± 1x10^ Pl^^Pa'^^s'^ '^ ^3x10^ 

^' . ... 6/ixlO"* <.8xl0 Pj P2 P3 1 128x10^' . 

90xl0^:i 3x10*^ Pl'^P2'*^P3*^ "150x10^'^ 

P - Pi ■ ' ■ ■ . 

0 P2 1 5,000 

f\ ^ - ' ■ ■ , ■ 

• 1,000 jc p^ £ 3,000 

The algorithm described in this paper* 
took 1.8 seconds to solve the above model, 
pficea and predicted sales are shown belovt 



$9030.75 



. Optimum . 2 -j .5000:00' 
Prices, ■ » 



3 



1526.94 



Sales for - 
these prices' 



A5,000 
128,000 



93,921 



Net revenue, generated from this short term 

\ 9 

pricing strategy will be $1,190x10 . 
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DISCUSSION 



In this paper we have presented] an 
efficient algorithm^ for the solution^of 
monomial problems.' . Certain, comments . . 
should be made here concerning the 
algorithm presented. (1) .Although j;he 
program of the algorithm . was written for ' 
polynomial objective functions this is not 

-necessary. Any dif ferentiable objective^ • 
could be easily sxibstituted for the one - ' 
chosen in this paper. The one used* 
however is totally general within the 
framework of polynomial functions. . 
(2) The program was written to solve .* . 
both constrained and unconstrained 
variable problems, it could easily be '/ * , 
modified for upper boxinded variables . 

"without the' necessity of using them in* the 
constraints.' (3) As was stated earlier in-, 
the paper all the typical linear programm- • 
ing techniques can be .easily -modified to ■ 
fit within the* f ramewof^ of monomial ^ 
p!rograms. ■ \(4) .The^ algorithm itself " is 
very similar to Zangwill/3_. convex .simplex • 
method .[9] and Wol^ej's rediiced; gradient , \ 
m^hpd [7], and. ,[8]-. However ±n our ^ 
algorithm no transformation are required . . 
(.5) . ComputaticTnal results .from^'this . 
algorithm are- very encouraging particular- 
ly in the light of possible : computational 

• improvements which can be made within the " 
linear' search routine.. 
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COST MULTICOMMODITY FLOW SUBROUTINES 



Hoang'Hai Hoc : " 
Cepartscnt of Electrical £sigineerijig 
Ecole Pol3rteichnique, Montreal, Canada 




ABSTRACT 

Several, mathcnatical . prbgramjning- formuiati^ of 
nonlinear cost multicocmocUty flow problems as veil 
, as a few of their applici^ons and solution algori- 
thms are considered, TTiree algorithms .are imple- 
mented' and tested^ od moderate size problfims5'''they . 
^ lye adapted Frank and Wolf e' s~method, I>arf ct^^ - 

KT^cit path flow exchange procedure^and 'speciali- 
xcd o^vex simplex sdgorithm, - Experimeital results 
seem M^iindicate that Frank asd Wolf e's method is 

^ competitive wh6n some what less preci- 
se solutions\^are accepted. The specialised convex 
simplex algor^.thm is particularly suited, to "post- 
optimization" tasks and is computationally superior 
. to the c3cpLici/t path flow exchange procedure, 

.1. TOTvPROBLEM AND ITS APPLICATIONS 

We consider tnti^ following nonlinear cost multi- ! 
commodity flow problem. Given a network with n no- 
des, let (1,2,, •,,p}, p^ n^^ denote the set of ^ i 
nodes vb±di are either supply points (sources, ori- 
gins) "or demand points (sinks, destinations), or 
both, -In addition, every node in "the network can ' 
be used as tprahshipment point. Let A be the set' • 
of -arcs in the network and D denote a p x p matrix 
whose (i, j) entry indicates the nonnegative number 
' of units iiMch miust flow between nodes i and j. 
Let the cost of shipping x units along (i, j) be 

where the cost is^a function of the total \ 
flow ilong. the axe. Let us use x^. . to. denote flow 
along arc' (i,3)* with origin s> the^Aciticonmodity 
flow problem/which we address is tiien 

Minimize f(X)« S ^ij^^ij^ ^ ^..r(^ 



of th^nailticommodity flow problem, A more intuiti- 
ve form of the flow conversation equations (2) can 
be obtained by considering! separately flow x . 
along arc^ (i,j) and associated with oxrigin^s an^ 
destination t. We have then the one (destinaition:) 
to one .(origin) node-^c formulation , of the multi- • 
. commodity flow problete. 



(i,j)€A 



^s J 



subject to 



Il(s,J)+ ^ 
Ic»l 



i=l 
/ (i,j)€A 



ij 



(2) 



for aU j = 1, 2^ m, s == 1, 2, p, 'j s 

s (3) 
, . '' ii ^ 0 for all {i,j)€ A, s = 1, 2,.'.., p 

■ ■ .. . " s . 

We remark that (i)-{3). represent vAat we call many 
(destinations), to one (origin) node-arc forinulatioa 
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MjHni.inize 



(i,j)€A^J (±,i)€A'-^ s,t 

subject to 
* i>* (i,j)€A* ij k^(j,k)PA jk 



-3)(s,t),j === s, j =1, 2, n 

0, j y s,t Sjt = 1, ,,^',,.-p 

D(s,t), j = t, t s 



(4) 



(5) 



St 



ij^ all (i, j) € A, .t,s = l,2,..,.,p t?^ s (6>^ 

Howey.er, the- most' intuitive fornrulation* of the pro- 
blem considered is perhaps the arc-chain formulation. 
This f oraralation, IS derived by providing, for each 
., (s',t) commodity, a-'set Pat paths from node s to 

node t and path flows x^ for all q 6 P , 

■■ • ^ St 



Minimise 



f.; ( Z a^?^ x^^ ) 

q 



f (X) = r . « « 

.(i,j)€A s,t,q 

SUl?j€Ct to 



(7) 



^ ^^^o"" ^(5,tr) all s,t - l,2,,,,,p, s ^ t (8) 



q€P 



St 



St 



0 all q€P s,t = l,2,,.,,p, s ^ t /9) 



where 

stq_[l, if arc (i^jjhb^ongs to path q € P 



St 



Id, otherwise. 



(10) 
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The nonlinear multicommodity nc"Cwork flow for- 
. mulation"^ presented above have beeri used as mathe*-* 
matical programming models for interesting and . - 
practical problems in transportation networks, in 
packet switched computer commtmications networks, " 
etc. We call the reader's attention to . two specif ic 
applications: the equilibrium traffic assignment 
problem in a road network [1,2^3,17,19,30] and the * 
optiinal static routing problem in a computer commu- 
nications network [8,13], . A more general multicom^- 
jnodity flow problem based on the arc -chain ^formula- 
tion mentioned above and including continuous arc 
. capacity augmentation can be found in Reference 
[14], where arc capacitieis are explicitly accounted 
for and the total cost is not necessarily a function 
separable or not, of only total flows ^n each arc. 

2« THE SOLUTION 'ALGORITHMS 

It is possible to present various algorithms for 
solving the nonlin'ear cost multicommodity flow *pro- ' 
blem considered within the general framework of " 
cither Zoutendijk»s feasible direct:ion method. [19] ' 
or the Conditional Gradient Method^ [14], Since' our 
principal interest is testing specialised;, algorithms, 
we choose to restrict ourselves to the fir's't alter- 
native. Then, -the feasible direction method for 
solving the problem of minimising a continuously 
diff ercntiable function f f rom R^^ into R over a con- 
vex polyhedral set F = [X^R^ 
cah be stated as follows.. 



Algorithm A-0 

(1) Find an ini-tiii feasible solution X°€F, Set 
* k 

(2) If X does not satisfy the stopping criteria, 
then go 'to (3). ... c 

('3} F^d 'a -descent dir^tion d^ wish re^spect to 
. X , iep. a.vector d €''R^ such that 

(X^ + td^) € F, for all t € [O, a], 

f (X^ + td^) < f (X^),. for all t € (O, v) 
k 

(4) Find a solution t to. .th^ one-dimension search 
problem 

. — Minimise {f(X^ + td^) |t€ [o, c]} 

(5) Set Xr:. = X + tV, k = Ic + 1, and go to (2). . 

A do^e examination of the algorithm. A-0 reveals 
that the determination of descent directions plays 
on important role in characterising different 
specialised versions of the feasible direction me- 
thod as applied to various* problems. A simple way 
to obtain such a direction, which consists in linea- 
rising the objective function^ was suggested by, 
Frank and Wolfe [7]. In fact, let be an-:optiinal 
solution of the following . linear programming problem 



then. 
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d = y - X 



'^"defines a descent direction with respect to X*^, 
provided that is" not an optimal point. 

The first specialised version of the^ feasible 
* direction method for solving the nonlinear cost . 
'multicommodity flow problem can be obtained "by 
observing that in thi^ particular case the L,P, 
problem (11) -is equivalent to computing shortest 
r path flow with respect to the distances' 



for all (i, j) € A 



3-3 



The specialised algorithm obtained [2,8,17] can* be 
stated as follows. , 

Algorithm A-1 • * * 

(1) Xet the current feasible solution be^ 




(i,3) ^ A). 
df. 



(2> 
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J, - and 
X, ,= X . , 

set Y_ = 0 for all (i,j) € A. Set s == 1. 

Find the 'Shortest path between node-rs and 
every node in the network using the- C^., 's as 
distances. 



(4) 



(6) 



(7) 



For each destination t in the network, set 
= + D(s,t) for every aire (i,j> on th^ 

shortest pgith between node s and node t. 

If s is not equal to the number of origins, 
set,s = s + i and go to step 3j^. otherwise go 
to step 6. - J ' ' ' 

■ k - - ' , " '/ 

Let y = (y^ |(i,j)€A), Minimise the function 
,f Along the^ine segment between X^ and y^, > 
'usifig:,!^. one-dimensional search technique. . 
Let; the HiTrn'TnTTSyi^ point be 

Test the stopping criterion, and go to step 2 
if it fails^ otherwise, stop. 



Minimise £f(X^) + vf (X^)^(y^) | y € f} , 'OJ^^i^^ \ 



Another way of obtaining a descent direction at 
a feasible- solution X is described below. It is 
more involved than Frank and Wolfe^s procedure, 
and very similar to the simplex algorithm for L.P. 
problems, . 

Let Q be a basis of the constraint coefficient 
matrix G. ,By partition niiig the variables into 
basic and non-basic sets, indexed by I and J respec- 
. tively, and renumbering them if necessary, one can 
choose as a feasible direction any direction 

•■satisfying the following conditions:. 
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4j i. 0 for .all j' € J such'^^Jxat x. « ' ^ 

• ^' " (12) 

=5 0 for m* i such that = 0 • 



and 



Choose 



C7f.(X)j - (Q-^R)^ VfiXy^f dj < 0 ' (13) 



r* = 0 



rj =*^(X)j ~ (Q^'^R)^ yf (X) J 
r. ■?» min tr.} 

• h::- i 



r. X. — max {r. x.} 

' "-2 "-2 i •. ; 

'oae can, obtain Either a descenf.direction of redu- 
ced gradient .tyge . . . 'O' , ,- ■■ 



d.. .= < ; A . : ■ 

.3 -r-Xj , -otherwxse; 



for all j€j 



(14) 



or a descent direction of. convex simplex type ' 

if j = i, . • 



■f: 



d ^ if |r. I > r. X. 
* J- 



2^2 



d J otherwise 



-(Q~^R) d^^ 



(15) 



. iinder the -f olio w±ag -assumptiofi: • r 

. ■ ■'■ . ■ ■ , • ' - ^ 

(a) r Xj ^ 0 -(the basis Q is non-degenerate) 

' / • ■ . - ' ■ " • ■ 

or * ■ ■ . ' ^ - 

(b) fer ^eryri € I, < |J for all j 6 J. . " 

la the following algorithm'we will restrict 
ourselves to the reduced gradient descent direction 
and the arc-chain formulation since the convex 
simplex descent direction is a special case of the 
reduced gradient descent directicfti^ This algorithm 
solves a series of restricted problems defined over 
a series of restricted sets of allowed paths • Jt 



uses a sequential optiinisation approach obtained by 
decomposing a restricted problem into a series of 
subproblems, each corresponds to an origin-destina- 
tion (O-D) pair. Successive restricted. problemif 
are obtained by the path (column) generation tech- 
ni'que des<?ribed below [16,19], • . . 

Algorithm A-2 " . '3. ; • 

•. ' • ' ■ ■ " ' 

(1) Let be given for each 0-D pair; (5^ t) associa- > 
* ted^with a po^tivc D(s-,t) a restrictwi set 
of allowed paliis \^ from node s to node t," . 

and an arbitral^ path flow, {h^^ 1^^^°^ 



.Set 1 = 0. 



(2) Solve the restricted multicommodity* flow J^o-- 
blera over the restricted, set of allowcd^ths 



{"s,t) 



s,t 



as follows. 



.(2.1) Seti^=:i, i^-=0. . ^ ' 

(2^2) if i, ^ is equal to. the total nUrab^: of 
opt..; ■ 

0-D pairs M, go to (3), Otherwise, 
set i^== i^ + 1, k =^ i^ modulo. (M) + L, 



(2.3) Solve the subproblem associated . with the J 
kth o-D pair (s,t) as follows. 

(2^,3«a) Compute 
\ . \ df..(x..) 

M for all (i,j) € A, 



c. . 



u .= 



dx. , 

^3 

(i,j)€A- 



a^.5? c^. , for. all q€P^ . 



mm 

q € p, 



St' 



(2^3,b)' Compute 



- u^) , if h^>€^ and 



> € 



2, 



.0, otherwise. 



S,t 



(2,3. c) If then go to (2.3.d). Other- 

wise, the floys on paths in P satisfy, 
the optimality conditions.. lien, if 

^ < 1 let i ■ = i ^ .+ 1 and go to 
' • opt . opt . b 

(2,2); otherwise, i.e. -t,^ 2, let. 
4^ i,pt-= 1 and go to (2.2). 

^ ■ • •, • . ■ - . 

^ (2.3. d) Search for t^ miniiii sing -* * 

> 2 f..(x. . + tw. .) 
- ... {i,j)6A iJ iJ iJ^ 



where 



and. 



St 



stq 

ij 



St 



O S t S mln(- 



1 



(2*3,e)- Lct^h^^^ h^^ + t^^d , all q € P , 

• r . .q. . q , . q' > ^ * St' 

'+ tt. j> ;all A," 

t , « + 1, and go to (2,3 

<3) For cyery 0-D pair (s,t), find a shortest' 
\ path b^^ from s to t with respect' to the 
.^ distances • ' 

- ' c . - ffijj^i^ ^ for all (i,3) € A. 
. ^3 



Let.u ^he tHc length bf-b: ; and 
maxiiBim f cngtii'f cr ^"iV significantly 
paths in ' P*^^. ' - ; 



z ^ be the 

St. . 
positive flow: 



Set 



St 



P 3^ tJ {b„J, if 



z - u 

^St St 



st^ 



St 



^st * otherwise* 



U P 
^^t 



St 



If ^^^'^ C then stop; otbernise, set = ^ + 1 
and go' to (2). \ 

GLose examination of Algorithm' A-2 reveals th^t 
it would be a cumbersome, if' not clximsy, book- 
keeping task to store all positive flow paths, 
check and to update all patt flows, -separately. 
Storage requirements could be fantastically large, 
even for mass storage devices, if the networks 
considered are of considerable* size. Fortunately, 
there. exists an equivalent, but very compact repre- 
sentation of path -flows, based on the node-arc 
formulation of multicoflfflnodity flow prolyl cms • Moreo- 
ver, time-consuming -shortest path calculations can 
-be avoided compl'etcly by using a gcnerciliscd version 
• of the augmented precessor index' method f^-] to., solve 
the flo<^ sibprxsb with an. origin node 

k^ while m ai n t aining all . other fia^ constant. These 
considerations have' given rise to the following 
algorj-thm fBO]. , ^ 

Algorithm A-3 * 

(1) Fpr- every origin node k, .find arbitrary flow ; 

(i,j) ^ A) and spanning arboresccnce 
E(k) rooted at k. Let i„ = -1 and i =0. 

T 'Y opt 

(2) If i^p^ is equal to the nui^er of origins Ng, 



(2) then aa optimal solution is attained; Other- 
wise, let » i^- + 1 and k - i^ mofiilo (N^) +1. 

(3) , Solve the fl<» problem associated with the * 

origin node k as follows. ^. 

(a) Let — 0 and compute the distances 

° 2i . for all ^ A. 

-(b) Cksmputc for >each node i ^ k the l^th of 



thc.path J. . from k to i in E(k) : s.»/ Z.^ - c 



uv . 



Compute for each arc (i,j) )? E(k) the reduced length 



(c> Defiae; 



^ I2 t(i,JX^ E(k)|~i>€2 and > ^] 

If either Xj^ or I^.^s nonempty go to (d). 
Otherwise; the flows x^. originating at satisfy ' 
this optimality conditiois. Thus, if < 0 then 
let i^ = i + 1 and go to (2); Otherwise, 
let i^ = 1 Sd^ go to .(2). 



(d) Betermine ^^^^^^^ ^^2'^2^* 
:r,^] and r^ ^ x. \ = :maxtr 



r j = min[r ] and r. 
1, 1 ^ ^ uv 



maxtr" x 
2''Z 72"' 2 ^, 



If r.,.. 



and a = +1;. Otherwise let (i, J) = (i^, j ) and 



a = -1^0. 
Retrace the cycle 



Let 



"0 (u,v) j? Lj J . 

a (u,v) € t^-Tj for all (u,v) € A. 
-or (u,v) 



uv 



( e) Determine , 

r niinfx^ l(u,v) € iTr- if a = 1 
^- k . _ i ^ J 

I minfx^^ | (u,v) € l\ a ^ -1 

If cr > go to (g); otherwise, perform the fol- 
lowing pivoting operation. 



(f) Let E(k) - E(k)U{(u,v)3-t(i3, j,)3 where- _ 
'(u,v) is an arbitrary clement of the set , 

_ •l3- X(«,v).^E(k) .1 r-iy^^^^>4^ ^ . 

(If is empty replace^i^, j^) by its sucessor on 
L*V J or £j J and determine again a new I^, and 
so on) and go to (b). 

(g) Find t^ minimizing 
■T - "f . .(x.\ + td. .) for t G [0, c] 



^3 



(i,j)^Ljj '^^ 
(h) Let ^ 



.X - = X + t^d ♦ / 
UV UV uv^ * 

c "= f (x ), for all (u,v) G vLr t 

UV UV UV^ , ' X J 



Set t '= + 1 and go to (b). 

3. ALGORITHM BtPUEMENTATION, TESTING AND USE 

- ^ . • *. 

In this section we discuss the implementation"^ 
of a basic version for each of thfe algorithms -Al, 
A2, and A3 presented in Section 2» Basiq, versions 
of 'the algorithms were programmed and tested on 
* relatively moderate 'size" problems. Th^ tests* were 
planned tt) gain insight into the .comjxitational . 
behavior of the algorithms, ' 

(3. a) For test problems on networks^.with abbut a 
hundred nodes and a thousand, arcs, the implementa- 
tion of algorithm Al does not present any serious 
problem. At iteration k of the algorithm it is* 
necessary to find the shortest paths for all 0-D ' 
pairs with respect to the .distances equal to the . 
marginal costs of traveling on arcs at current 
flows X^. Shortest path flows are obtained by ^ 
:>--v:^r - for every 0-D pair (s,t) the flow requi- 

3(s,t) along the shortest path joining s 
ar.r t. Floyd's algorithm^ [6]. has been used to 
determine the shortest paths between all ipairs of " 
nodes of a network.. This choice has been made in 
order to facilitate programming tasks. Computing 
time could be significantly reduced by repeatly 
using Dijkstra^s algorithm [4], especially when the 
number of origins N^ is much smaller than the to-, 
tal number of nodes. The Golden. Section method 
[15] has been used for * searching a mininum>of ".the 
function f along the line isegmcnt [X^, Y*^]. The. 
algorithm is started, from the initial -shortest 
path flows with respect to- any estimated distances,. 

Two test networks are considered: one is' the 76 
arc, -24 node network of the City of Sioux Falls, ^ , 
South Dakota [17], and the other is the 376 arc, 
155 node. network of the City of Hull, Canada [19]. 
Glob«d computsfctibnal results are presented in Table 
1 for comparison With the algorithms . A2 and A3. 
We remark that for the basic version implement^. 



computing time 'per iteratiwx of Algorithm Al is 
independents of the number of 0-D 'pairs. Conse- 
quently, f oi; the Sioux F,alls» network, we' solved 

■ only one test jproBlon with 528 0-D pairs. Moreover 
to 'reveal the ^nvcfgcnce speed, of 'algorithm Al, ue 
have plotted* in Figure 1 the Value of the objective 
function, maxumvn percentage change in' flow compo- 
nents, and computing time as functions of the • 

. number of 'iterations. 

A converjgcnce dif^iculty . inherent to Frank and 
Volfe»s algorithm pecurs v»hen the objective functi- 
' on assumes a minimum point in the interior of the 
feasible region. In such a case, the direction 
indicated by the gradient may change very* rapidly 
as the sequence ^f points approaches the opti m al 
solution. • Consequently, the points -m^iy sig sag 
. around the optimal solution. In our test problems, 
although the -gradient of the objectiveVunction 
can^not be zero at the optimal solution (since the 
objective func;cion is strictly increasing and has 
no ^^tationaiy point in the feasible- region) the 
convergence "seems to slow down appreciably when 
approaching the optimal solution. 

. (3.b) The basic version of algorithm A2 is essen- 
tially the computer program given in [16], imple- 
menting Dafcrmos* algorithm using a column gen era- 
'tion technique. In "this variant of algorithm A2 
the descent direction in (2.3 b) is actually repla- 
ced by the following* 



' • V u = min (u | q € P ) 

a 4 ^SL ^ . 



. d ^1 

.^■'^ ■ a. ■ - 



= -1 



d^=' 0 for all q P^^, q ^ a, q b 

An appropriate move along d means shifting flow 
from the maximum cost path to the minimum cost 
path in order to equalize travel costs on these 
paths. . • 

The program has been modified in order to acco* 
modate much more 0— D pairs than the maximum number 
of 20 -originally allowed. , These minor modifica- * 
tions -are straight-^f orward and consist in reducing 
the core storage required to memorise* path ^and 
path flow data. This reduction is easily- achieved 
•by performing flow adjustments fqr one 0-D paif" 4t 
a time'. Only, data associated with the 0-D pair 
considered need to be in core*. When all path flows 
for this 0*-D' pair * satisfy the optimality conditions, 
Upda-^^d data are written on disks for latt.er, uses. \ 

(3»o) The essential part of algorithm A3 consists 
in solving an one— commodity flow problem associated 
with an origin node. This i^ achieved by -a gene- 
ralised version of the augmented predecessor index 



method, -For strongly connected networks, it is al- 
so possible to work only with a basis which can be 
^represented by a spanning arborcs.ccnce .E(k) rooted 
at origin node* k. For the arborcscei>c« rejprcsenta-- 
tioa, we use triple-label notation [12] in which 
each tiode i receives three labels 

. . child (i) pointing to first node j such that 

(i,j) € E(k), : ^ . 

- parent (i) pointing to the node j for v^iich 
(j,i) € E(k), and ^ ; 

- sibly (i) pointing to node j i such thaC 

. (parent (i), j) ^ E(k). - . . 

Node numbers z, are then -readily> obtained by ' . 
traversing* arbor esccnce E(k) in end-ordei^. Reduced 
arc lengths computed from node numbers and marginal 
C9sts permit us to verify- the optimaldty condition-s/ 
and to choose an 6W=^r^:^cHter arc for uiiich some 
flow adjustment is performed,' 

Let (i, j) be the out-of=-kilter arc considered, ' 
and let us suppose that -the reduced length rr- -r is 
negative ♦ Thus, flow adjusjDnent consiists in'^t^-ans- 
fering flow from (q, r, s, J) to (3, t^ i) Xo- 
reduce -r to zero. The same a^djusbnent process ; 
as that usid in [16], i,e. an. in,te^:;vai halving ^ ; 
method, is adopted, >^since^ in this -prgce^s no itera- 
tions are required if integral (lump) flow transfer 
occurs, Thisyspecds up, the' computation,; expecially '. 
at the beginning of • the solution process^ and' gives 
bettfcr results than the golden section metho'd, ' 




Ba sis. /flanges are frequently required* To per- 
form a basis change, a zero flow'arc (r,s) in E(k), 
is replaced by a positive>flow arc (u, s). Such an- 
arc (u,s) exists necessarily, if flow on (s,j) is 
positive^ Otherwise, .(s,j) is considered instead 
of_(r,s), and can'in fact be replaced by'(ijj)« 
After a basis change, if arc'(u,v) isr inserted into 
the new arborescence^ then only sub— arbor e scene e 
rooted at u needs to be considered for purpose of 
updaticfg node numbers." 

* > ' ^ . ■ .■ « . 

To start the algorithm, initial arbore^cences 
rooted at different origin 'nodes as well as arbitra- 
ry initial flows are required. We calculate the 
short e:5t path from ah origin to ever3;^node in the 
network by means" of Dijkstra's algoritftm-. At the 
end of the shortest path calculation we obtain the- 
shortest path flows; an*d for each node tfie shortest 
.distance as well as its' predecessor node on the 
shortest path from the origin to the node itself. 



This does mean that .we have for each origin an 
arboresccnce represented' in predecessor index nota- 
tion, which is easily transformed into triple-label 
notation and used for basi^; representation. The 
shortest path flows obtainedcare used as initial- 
flows.. ■ ^ " * ' ,. • . 

(3.d) We will now discuss experimental results 
obtained frj^m solution of the test problems by ' 
means of algorithms A -1, A-2, and A-3. ^ 

First, it is clear from the statement of the-- 
algori'thms and presentation, of the basic vcrsioiis 
implcm'ented of that aigorithms A-2 and A-3 are quite 
similar. For both algorithms w,c may interpret the 
underlying itterativc computational process udthin 
the framework of traffic equilibrium as. -a process 
by which rational drivers acquire knowledge about, 
their travel cost, switch to cheaper routes, conges- 
ting them, and causing a new flow pattern to en- ^ 
volve. This interpretation gives rise to a realis- 
tic stopping criterion for .algorithms A-2 and A-3, 
which requires that the most expensive and the 
cheapest used paths well as 'thp cheapest unused 
path** frora.'ari origin t(^ destination 'rao-st not? 
.differ^ more than, say, ^^in travel cost.! Such 
stopping criteria may also be used^to^ generate 
solutions simuiLating a well accepted i;j?ct that .ck^ 
of drivers dof not care for differences of .*J$?jul 
travel cost ^etwc5en*paths from- their origin -to 
their destin«ioa^. " ^. v ' - ' f 



V' 



Second, although flow adjustments in algorithms" 
A2 and A3 are^ quite similar, algorithm A3 uses a. - 
more compact representation of flows without ^ex-i '''' " 
pliciUy storing ai paths allowed. Thus, we may ■ 
expect algorithm A3 to .provide better perf orm^cSs 
thaij algorithm A2'in terms of core and mass storage 
requirements as ^well as computing tijne since less 
book-4<:eeping is necessary and the shortest path 
calculation avoided. - The experimental results ob- 
tained confirm this conjecture in a very striking 
manner, i^igure 2 shows a relation between corapu^. 
ting time and number of O-D pkirs for a series of - 

.4 Sioux Falls network test problems presented in 
Table 1, The compting time^of algori,thm^A2 is 
sevw times greater than that of A3. All runs were 
executed on an IBM/Systeiii 360- Model 50- with Ampex " 

ECS under OS^fVT, using Fortran IV Level G Compiler. 

Third, the <:omputational behavior-of algorithm- 
Al is' significantly different from that of algo- " 
rithms A2 and A3. Defining an iteration as a 
shortest, path flaw calculation (essegtiallv^an ex-' 
ecution of, Floyd's algorithm) followed by 'an one- 
dimensional search, the value of the objective 
function, maximum percentage change in flow compo- 
nents, and computing time are. shown iteration by 
iteration in Figure 1. The results indicate .that' 
improvement per iteraxion." is ver>' substantial aJt ' 
the beginning of the iterative process, but that 
the rate of. convergence slows when, approaching thtf 
optimal sofutiQn. This means that increasing the' ' 
'precision of the solutions is ^tim'e consuming task. 
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Suqh: a characteristic cou3d T)e ermoying for people 

• • attempting .tp use algorithm Al^ in selectilig net- ' 
i. work tppolbgy or network improvement projects. 
■ • . ' y.J' , • •■ ' . ' ■ " . "■ 
' .(3.c) . Jife are in fact "searching for an efficient 

' ^DLCtwork 'configuration evaluation procedure' to be . 
used ii selectinjfe'networfcrimprovenicnt; projects sub- 
ject to a biidget constfiint* * For a regional; rbsii 
^ network^ vihere congestion may reasonably be negli- 

* gibleV we have , proved a superadditive proper^ of 
a natural objective function and devised a power-- 
ful -branch search procedure [10], which /was later '^^^ 
greatly .imjarovcd ancr reported in [5], 'Interesting, 
results presented in ts]- also show that hGiristics, 

' related- to those suggested in [10], give solutions 

' cxtrem^y close to the optimal ones/ Ve are 

la-esentljy devdopping similar procedures for net- . 
works' where congestion is no longer riegiigible [11] 
In such a case it is.nece^ary to perfonn po^t- 

. optiinality analysis ty rtooving a link from or ad- 
ding a link to a network coiniguration# Algorithm 

r /A3 seems particularly suited to this rcoptimisa-"^ ^ 
tion task. Table' 2 .shows xcSnputing time .required 
. as- function of link insertions/deletions. " " ■ 
^ \ 'r . • • V ■ C * 
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ABSTRACT 

In certain linear programs, for example^ ^^ose - 
etrising out of production- inventory type problexas/ 
a large number of constraints have a spec^^ form* 
Examples are: simple upper bounds, generalized 
upper Jx>und$, variable upper bounds^, and general-, 
ized variable upper bounds. This paper identifies 
a . class of constraints which enables us to obtain 
a large triangular subcoatrix in any feasible basis i^^. 
Also presented Is a method for representing these 
constraints, implicitly within a modified version 
•of 'the. revised sin^lex method- ' ■; 

1^ Introduction -. //. 

From thV early fifties on; there has been a. steady 
progression of optimization strategies and tactics - 
for solving large linear programs -JL. Such large 
LP's are* common dn problems in- production and . 
operation management.^ Exainple's '"eLre those derived . 
froia mixed, integer linear programs 'Arising but of 
such problems, as production-inventory planning ■ 
'and- facilities location. -Consideration of multi- 
^ periods and/or imilti-pfoducts increfases the size 
pfN^ese problems, substantially- Another .factor 
^cbnwibuting to the increase in .size . in so^e 
"problems is the possibility of tighter 'formula- . 
tions 6^ integer programs as opposed to lc?ose.- ^ 
formulations. . Impo^iijce of such tighter 
formulations is- well . explained by Schrage 
(1975b), . . ^ ^ ; ; 

One area of research on large LP' s is ^that of 
-. tactical variations in pricing and pivot 
selection^ Another area, which is the areS of 
our Interest, is the . exploitation 'of special 
'^tructuiT^s" to* reduce storage and computation 
time by avoiding explicit consideration of ^. . 
full inverse. E^janpies of such special struc- . 
ture constraints which have Seen represented 
implicitly are simple upper bounds (SUB) , 
generalized upper bounds (GOB) / and more . 
recently variable upper bounds (VUB) (cf< Schrage • 
(1975a)), and generali-zed variablfe upper bounds 
^(GVUB) / cf., Schrage (1975b) i Mention should also' 
be made* of ajigulatr structures and factoriza-tions 
.';:escribed in.Lasdori (1970) and Graves and MacBride 



For a recent review of some of this research see 
Jeale* (1975) and also the preface of the same 
ioblication/ * 



This paper identifies a class of constraints- 
appearing in certain dynamic production management 
problems which allow us., by judicious row and . 
column permutation, to obtain a large lower trian- 
gular submatrix in any feasible basis of the prob- 
lem. Section 2 introduces these constraints and 
illustrates with examples instancy where such 
constraints occur-. Section 3 discusses the ' . 
importance." of having a large lower triangulair ^.sub- 
matrix - .Section .4 outlines Jthe.. detailed develop- - 
meht of a solution method based on^ a -modified • ■ 
version of the revised' simpleii method; Xn^lement- 
ation considerations -currently underway 2ire also 
.discussed- . ^ ; \ 

■2. Triangularity Constraints " ^ ' : • 

In this section we define triangularity constrainll 
;and'also.givevSome^,examples of where they arise. ' . 
7txs^ define- ■. . ^ ' \ ' 

Eligible set of row i^^ the variables - 
. including •s.iacks"'^^ that have a strictly posi- 
tive' ■cde'fficient on the iLHS of row i;. i.e. 

.a;, > 0].^:-^ ^ \ ^. . ...'^ . ■.. 



Now we define triangul^ity . constraints . 



A set of constraints 'of ^ linear program is 
called triangularity constraints or T 



constraint** -: 



(a) *^eir; RHS are nohnegat^ and 
> ' the .set of constraints can be per- 
."v . •muted so that in a feasible ]:>asis 
^ *^ .'^ to the problem, a vaxiaJ^le in the 

: -. eligible set of»any T constraint 

does not have a nonzero element in 
, . . any other T constraint lying eibove 
this constraint.. ^ ' . 

'Example 1 : One^chine, Multiproduct ^ Multiperiod 
Scheduling "Problem with Setups. . • ' 

. " . • I ' ' " 
. ■ p T p T-1 ^ :r 

Min E 2 k 2 + Z Z Z (j - i)h^r^x. 
* ■ p*l t=r. ^ ^ p=l i=l j=i+l . 



11 . 
P P 



■ , V; - Ci).; 



v..y..\.- 



all p, j 



all 



strairvts is that we axe able to get any 'feasible 
basis into the fonn^ ' .* ' ' , 



V C3) 



f4)T 



(8> 



ott y • - y ' V-. 2 <.0) 
.. , ip. i^l^p ip — . 



X. . < y. 



13P - 



all i, j, p 
all" -i, -j, p' 



where Tj^^'cqjnposed of constraiat^ (4) and (5),..! 
and isj^vfer* triangular* Note that in order. ro- 
get tlSfe "form we had to .place (4) .and 1[ 5^ /in-^-tii^' 
order at the bottom of the constraints^ / \i 



(5)T 
(7) ' 



■ ^Example -^i Dynamic Plant Location- Proble^Xi 
man. and Schwarz r 1974} * - • 



Mm Z F.y. + E c. , x. . "fc 
. ^ it-'it . / ^ lit i^t c 



^ where 



d^^ = demand for product p ■ i^^gpriod - 1 

. .-r ■» -units of'-p" which .can "be produced in ■ 
•. ? :'.peri"od t'J -..-' , , ■ ' V. 

= setup cost for. product p. :• \_ ■ ■ *' 

• rf*-= one period .holding cost for product p 

X. fraction of ' production' capacity "in 
. ■ ' •period i used to satisfy product p^. 
; demand in period j . • 

if pro53uct p is produced in 
eriod i 

' iP* u 1 

else 

p.^v"if - setup.i^ made in period i ' ■ ■ 

V-'^eise.;/!/./.;:--';;- ; ■■ ' : ■ 

■ Note that" the ".-LP -.relevant ".to^-our discussion is'* ' ' 
the relaxationi ;4in. (l)/s.t. (2) - (6) , The 
letter, T against- constraints (4) and (5) indicates 
that they are T constraints. Now 'consider theses 
two sets of constraints together. 

•For any- RHS* constant' which is; zero ^ without loss 
we can assume it to be 'a , small positive number. 
"Thus one of the Vauriables with a positive coeffi^ 
cient in the LHS of each constraint of (4) and (5) 
must be feetsi]>le. basis. For these variables • ^ 

of (5) ,:: there are no other nonzero elements, in 
their: CO Itjmns within (4) and (5). This -is no ." 
.surprise as C5) is in fact of VUB form. In (4) /;" " 
the y^^. columns have no elements above the posir 

tive element corresponding to the row in which a 

paztticu-lar y. .is basic in a row of (4). Thus-**- 
ip 

.(4) and (S) qualify to be T constraints. 



■ s'.1:. " , Z X . , '-5? 1 



all :j, t 

/' -.' . ■ • 
all i, t 



od\ 

,(10) 
(11) T 



'^■-ijt -^it ' '. '^^^ 



y is integer .,(0/. 1) 

■ it- . . • ' ■ ; 



(13^) 



where-' 



F - fixed cost of v operating? plant i ^ Sn \' 
' period t - --^ ^ ., h '^ ' * ""■ 

c^.^ = cost of 'transporting a\uhi-£ fromi i 
to j in period t 

X. . = fractj-on of requirement at js^ supplied.- 
by i. in period"'*t 



3^ suppx. 



'x.t 



|1 if -plant- X is' "open" in period t 



0 ' else. '. 
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rimnediate advantage" in recognizing.- the con- 



" The-LP. relaxation' is:r- (9), s.t. (10) — (13) -. 

Now; -following 'arguments simil,g.r to those in -exam- 
„E1^- 1'. any feasible basis into form' CB)'r 

(11). "and (12) become the triangulaxity constrain tsf ; 

^ ' - - ' ' ' ■. ■ - ■ 

To summarize^ the motivation behind identifying" T . 
constraints- was the ability to. get any feasible 
'basis- into the. form (8) , -The, constraints, form • 
"the lower ^xikngular T, and in* the examples given 
.^abcve' it would be seen that T - is fairly' large..- ^ ". 
Thus; considerable savings could be anticipated . . • 
if these T constraints are implicitly represented. - 
To do this, we exploit some special properties, of t 
triangular matrices. - ■ 

3. Usefulness of the Triangular. Stimatrix:- - /^^'^''v^-- 



:^^lar. ; 

The following, properties of' ■ triangular matrices • \ 
make it , convenient to haVe a .large ^ lower trlangu-\ 



lar submatrix in the basis. 



' (i) la .Our problem' T is nonsingulaf;: PHh^s > 
: ' T : exists. -J vis '^^Iso lower triangi^^ : 
' : -v^- -i" - • • ' ' 

: Nii) T '.ca;^ be computed bjr, iterative substi-* " 

■/* 

; tution* . CIn- general,. T is riot explicitly . 
c needed. ). • 



vectors 



• y^iii)" Pre-mult^plicatlon ofKjaatrices or 'u 

-1^ ' ^T>^^ • •■ can. be . doiie^i-thout ^ tin4ing . T ^ . . explic 
. lifly."' For; instan5#r''^let it be .an n ^' m'- matrix 



F afid 



. . and we want F *. T R- . - If we partiti£)nv £ 
r^Aby-iolumns, * - . , 

■■*NOW if we solve .the system ' Tf7 - r, Ci = 1, 2, 

• " 1 i ■ ■- 

* - • • ■ ■ ' • ^ ■ - . -1 

. . n> /"^^ back-q^ubstatution we, cem get T R in 

mn-^- multiplications. In using the revised sim— 
. . plex method pre -multiplications of this type occur 
. "several- times/ * 

. : -■ Civ) 'Both (ii) an?, (iii) are simplified if T 
has further special* st3ructures , e.g., SSnd struc- 
tures'. • Some practical problems tend to have, such' ■ 
band matrices in place of 

.4- , Implicit Representation of T Constraints . ■ ■ 

" We saw earlier that for problems with T constraints 
a- feasible "basis could be -written in the form; ^ 



x/> 0 



*= ' 0 wi IL - be •the • top row • 



Constraint; - z\ Z -^f.^^^ 

(row zero), in the tableau *^ "If we had the* full . 

B**^*, the top row of this^^vould have breen the 
vector of dxial prices {tit} ."' Also,: let -m C<.M)., 



be' the last^expl ' 
are in rows m - 



T constr^Lints , 



Construction or the Prici-g Vector ' ^ . . 

All elements of the pricing vector - (tt^} can be 

cofflJ)uted. froiil .the ' '.r Her inverse and the. other 
.information we,carr/. If 'i then TT^;. is. 

simply element. .1:. -of ^roK. zero of -(C -DT.> ^\ 

TT^' starting' from 



For 



* we. can com^u-t;e 

i - M as follows.- Denote by b(i) the. column 
basic in row i. Then^ 



■ _ ^ ■ K U ■ ■ , 



i*e., IT 



mb<M) 



'for i.. -..M 1: J ■\«w,.(mJij. + To^iVliWl)- 



Jith tfJTs 



With tftis form of B, 



(C T E) 



we have 



: Ir -I ^1 



r"^(-E) (c-dt"."^e)""^' T*^.(«Ey(CrDT"S:)""^C-D).T".VT"^ 

. " " • * ' ' (15V 

^ ' ' : . ■■ .'.-■".•*. 

^We plan to carry only the sm^ler inverse " 

{C^- DT ^E) ^ and not tfie' larger B ^. Informa- 
tion in the coefficient .matrix- A would be; V 
carried in paurt .explicitly and in part using' ' -. 
.^pointers and link lists,. . The next step is to '\. 
"apply the revised simplex method to this reduced 
working basis. 'To*cfo.this^ we write ?the entire 
LP in standard form: . . ' 

s - Jlax z = - I a^ .X. 



j=l 



Oj' j 



s.t. 



N 

Z a. .x. 
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a • ! i '= 1,. 5, . . M 



In general for m 



m 



'-^ VMb,(M-l) ' ^ ^ 



- 0 



In fact, we do not always have to start computing 
from the last row. Certain '"'^'s* namely 



IT. 's 
1 



those corresponding^ to columns of T which have 
only the diagonal element nonzeroyare alsovdirect- 
-ly computed. This property is. useful in imple- 
menting the; pricing procedure. When we need a ■ 
TT^, i > m, but do not have the price of a later 

row, we find the column basic in that row and 
price it out.^'This loop will produce the required 
. TT . ■ as ' soon as ve ' reach a column with only .the 

■ ^ ■ ■ . ■ ■ ■ . ^ 

.diagonal Element nonzero. ' In -a typical implement- 
nation we 'would Drice *out only a 'small portion of 
the columns at any. iteration. If none of the 
columns priced have elep^nts in row i, then, 
-that. TT^ is not computed. , The fact that a^^ 

'are usually simple elements (0/ 1# -1) also helps 
in reducing multiplica^ons. • ■ ' \ 

Selectiinq/aji .Enfering Columry 

^ . M ^ ■ 

This is done by computing Z "^^^-v^ selected 

k=0 ,^ 



.jwnbasic.'coluians- onJ.y .saving here possibly - 

. would-be "by -using the Icnowledge that' some of the " 
si^^'s: are '.+1 • or ; 

Generating the UpiSated Representation of. the - 
.. Entering Coliimn' ~~ ~" .. ^ 

'Define the fbllovdng notation: 

f "V " column vector consisting of 



•• _ 0;ij"= element , i of a ., .■• i .-. 0,. 1/ ■.-. .> M.' 

, the- notation ii;k) when used as' a ' siibscriij.t . , 
■ '.. "<^enot€s. elem^nti- r i' ■ thrcrtigh )c ..'along the/associ- 
- ^P^^ dijnexisionVxe^g. , a..^ = ^ CCf-l^ 3 * ' ' " 

'- ..^^ j the entering column^ -'th'en wa. want • to get 
■ S. 3P j - d- , - Prom ■ ,the "pr ej?ious J.y en cbun ter ed 

form of; B./ , this givesV \ • , - ' "-^ 



. . Df . . In an iniplementation: current Xjj/under . way w^- 
plan to store :by columns .. - f . is quite speurse 

■. and we heed" access only those columns of -'D" ." : 
corresponding to the few' npn zeros of^, f. Because' 

f^*(m+l;M)j . .^^ a^colmm vector, , DT'^a.^^^^'^ ..• 
• IS easily computed." Recalling that ! ; 

(C - DT, ^E) is the smaller inverse, we are 
carrying, «• Q^^q^j j now available* " • * 

" Consider (17) simplifying' further^ ' * 



r r '3 (m+lrM) j ^ . 



- f - 't"^" 



CO:;m) j , 



We already- have f ■ and^ d 



(17) 



(C - dt"^e)'^.'^C-D)t^'^ +^T""^ja : 
Consider (16) ; "simplifying further, " . 



(DT computed easily if we use the 

special properties of -T stated in section 2. 
Let" ■ 



(O-m)j* Now -we compute 



'^(O-m) j^* -^^^ pre-^miltiply by T^^. Pre- ' 

multiplication; by. \ T - follows the simpler method, 
.aiiready discussed*"- ■ ■ .. , 



Qt/^, -f and hdnce the entire a-., is now 

, - ^ available.* / . ' 

Selection of the Pivot Row 

;Thi5 ■ is done' very much .in thesusual fashion by 
computing ratios. We already ni^je the updated - " 
version of the entering column, and we need the 

updated, RHS:- ■ ■ , 



•0 -0 



^/■« given by: 



Then 



■ f as T a 

Cm+l;M)j' 

• ■ - ^ ■■ . . 



Tf = a 



(m+l;M)'j 



and we solve this, system as in Section 2- / Usually 
^(m+l;M)j contains only a few •nonzero el^&ents, 
and we start" with the top-most nonzero element &n ' 
*Cm+l;M) j * corresponding: to it . gives Ae 

^ esponding f%**: The' next step is to compute..,;. 



e could follow the same steps -in -th^ previous S ' 

section and compute a,^ . but a better way 

. / (0.;m)0 ' ' ^ . 

to. carry a*^ \i' explicitly, and update each ^ 

pivot ^by consi^ring it as another column of 

"cc - DT^-^yE"^.^.--.._r: ■ ■ 

^(m+l-M)0 computed by back siibstituting the ' 
values Of vkriables basic in implicit rowis that 
.have nonzero coefficients in- a given explicit row. *• 

We need only the elements of ^ a' ■ , ^ which* * " ^ . 
- - ; ^ -. ^ (m+l;M)0 - . 

correspond: to 'Qt,i_^,\:'\ - f 0"., ''The actual procer-" .. 



• dure, to get the required \^^^y suDstitu-' 

; tion i-s similar 'to the" Icjoping* discussed in ■ ■ -i 
; constructing 'the pricit>g vector wJiere .the stair-7 

■ , case structure of Tlv ife- exploited. - ' : r 

Pivoting y . ^ \. . ' 

The t^pe- of pivot to be performed' depends on > 
whether -the" departing colvann is -basic;. in an-explic- . 
it row or an implicit rov. , Let u& , consider the 

■ se\osral' cases- that- may-arise* . . ; 

; Let / , 

• - c * entering column 

../• ••'.-■ ' ■ ' , ' •■ , 

, .r ^d-.*. departing column 

\ ' .r ^^rov- in wh^ is' basic- 

. ,C«£He ix) . Departing column basic iii an explicit 



The 'change of a column- in, (D,. T) does not effect" 

if; • 



•It affects OT ^a , 



^,{D;alb,(i) .".-'Z^' (m+l;M)b(i) 
column b(i) has a nonzero in ' E in any" row that 
ccamunicates" with row; r of T. In other words, 
there will be ' no ef f eW if ^ ^^.^,^^ (1) where 

q is the- largest nu^S:>ered^ row:in T. .aix3ve r/ 
which has only a lone nonzero element' Cthe +r in ■ 
the diago*n2Q) and such -that all a^^^ = 0 .for • 

k < q. The reason for- this lies in the fact that 
under these' circumstancesv the solution to • ■ " - . 



f.- a 



remains; tKfe /same eveiijafter ;the ch^^ T.^ . . 

Once we hiave-* identified a column b(i) for which 
all zeroes in 

put'e j' . 



'(q;r)ba}\ 



are not* zero, we com- 



Column . c replaces d ^in */ (C> .BW The chang^- 
- affectis only '-^C ' and - and -the effect 'is' ta • ' 

- ■ / • • •■ 

■ to change only- one column in (C - DT ' Ei. The new 




1- 



E) 



' - . . -1 , . . 
'^fO;m)c.^?^-^(m+i;M)-^; 



ipdated version of c/' -we; had 



column in 
In comput 



Thus. the new . (C -^"DT £) x's .obtained. by^ using. ; 

a, \- as" pivot column and pivoting on element. 
- .CO;m} C V 5. . .. ' ;- •. . ; - ' ' , ; . 

a *. Thus, this pivot is identical to a conven- , 
tional revised simplex pivot- 

Case Cii) a- Departing column basic in an implic- 
it row, ari5 c can directly replace d in row 



r^ 

a 



rc 



x.e. 
> 1. 



r > m and a, - 0 for m < i < r, 
. / ic A • 



Now a column is changed in (DT) * . ;,The difficult 

part in this case^ as well as in the next ceise, 

. (ii) b,.. is identifying which columns- of 

' -1 " • " " ' ' ^ " ■ • 

{C- DT E)' have been affected by this column 

\ change* ' ■ ■ " ' 



(^0;m}b(i) 



^ ^n'^n v^(m+l;M)b(i). 



'Let 0/n • as a subscript denote the old/new^ 
■ TQSatrix; e.g*^ T ^ is T ^ before change* of a 
^ column in (DT) * » . . - ■ ' ' 

■Let ''\ - ■ . ' : 



(C - DT^^E) 



'We- have to repivot in any row .i ; (i ..f IV r- -/,'" ' - ^ 
for which ■ * • •' " 



where ^ e7.. is a 0 vector except for a 1 in row i . r 

^ ' 2: . . ^ .■ 



using the new £md .T- Premultiplication of % 
this by the current ■ . R ■ / gives us' the pivot column . 
The pivot: itS'^raa^ -in -ifow^'i 

Case (ii) b .' Departing co 1 umn ■ basic in an imp lic- 
it row, but . "c,; cannot replace- d in; row' r„i.e., 
the entering* column not Of i^pecial* f orm as in case 

_ -Cii^^a: -. ■ . . . .'■ v., ■ ■ 

Tfiis means that the entering column" (call it t) 
cannot, replace \d'. - If it did^ it would destroy 
the' partial triangulztr structure- ■ As in <3VUB, we 

haiidle;^thi's'an two steps. / ■ ■ • • 

Step i . :Find a column - . c . cur r en tly^^ basic in an ' 
explicit row that can, replace d. Such 'a. column* ; 
exists , .because th'^ new basis—with t dn place^ 
of ' d~i^ a feasible, basis, and it too -should be' 
adjustable to 'form (8) . ' , " ^. 

Once- c. is found, step 2 is like case (ii) a. 

As many pivots ss there are columns changed in ^ 

(C - DT~H) would be nee^ded to ' update its» inverse. 



Step 2. NOW replace c 



by t. 



Pt 



This is like ca^e', (i) , and requires a piVot on a 

using a" ^ - as the pivot column. 

t r ^ ^ . ■ 

Though it is more complicated than the VDB and 
GVUB implicit representation, the implicit repre- ■ 
sentationof triangularity constraints also saves 
much computation iri the early steps of the revised 
simplex method. The pivoting step;, particularly* 
case.Cii) a, may involve several pivot operations, 
but this may. veil b^ compensated by the savings 
due-; to haying these .'operations performed in the- • . 
smailer inverse instead of ^tHe full inverse . 
Storage-wise, ." implicit- .representation is ■ efficient 
in that it enai^les, .iis'- to ^stpre^ problems' w^ • ■ : ■ 

large numbe r of triangularity -constraints entirely 
/in TBjaln - 'tiremtDry . ■ * . 
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ABSTRACT 



.. - ■ The "Zrtrahsf orm 'technique ' is applied to the' 
optiiaizatibn' -problem -consisting of a: quadratic 
objective function and linear difference equations-. 

: A rigorous, proof of the. general solution is offered, - 
and a coinpiitational: method is described whicfr is . 

■ iilcely iM-h^ more efficient, than matrix inversion 
oir^oatc^Jc, iteration 'appr^^ . ■ . . ■ ' ■ * 



- IKTROpgCTIOK ' . ' 

. : ' .' In a recent paper,. Hay. and Holt /4/ have ' 
presented a general solutioft for the -optimizatioh ■ ■ 
. problem, in which the ' objective fuinction is quadratic/ • 
and the variables ' are; subj ect to linear diff erence ' ■ . " ' 
' equations r utilizing ' a technique - known as^ th^. . - 
. "Z-transform.;/ ,This technique is an alternative . to . 

other . solution" methods currently in use, such - as 
. matrix inversion (e.g. Theil /7/ and matrix 

iteration (e.g. Bellman /l,p'. 78 et.seq./.. Chow 72/5 . 
- While-.the developments in /4/ have contributed some , 
' additional insight into the ^ class of quadratic- 

linear optimization problems, the Z-trahsform 
■ technique as' presented seemed to. have little 
computatioMl;j5lgnif icance,. for' as the -iauthors 
stated,.'"!. ♦ matrix iteration or matrix inversion 
are likely 'to be computationally ;saii]^rior approaches,.,.**. 
. /op.cit. p. ' - r- . - ^ 

" * ' » • ■ . r * ^ ' _ ' ' ' . * 

.... . , . ^ . ■ ^ 

" In tHis paper/ I return to the;.2-tramsform 
technique for solving quadratic-linear optimization 
problems. :I.-_pay particular attention to the ^estion 
of - computational feffficiency', 2md reach the conclusion 
that the .Z^tramsform technique can in fact lead to 
solution methods computationally sifperior to either 
matrix , iteration- o;^ 'matrix jlnversion, .at-^ least unfiier - 

' certain stationarxW jaissum^ions,-, contrary to what 
is suggested in /aK. , lu arriving ',at this conclusion/ 
.I.suqgest an algorithm that is different from that' ^ ; 
in' /4/, in the way certain characteristic roots are ' ; 
determined. These developments were made* '^is, part;of . 
an investigation ./3/ of an educational^ planning ' . ' 
problem/ where because; of the large r^vca^er of system ^ 

:.vyari,ables, , and; the long lead tijues involved (which 
result: in -.difference^ of- highjorder) / . ' ^j. 

computational efficiency is placed at a piremium. . , 

' Altogether. 'apart from the i^sue of . ' " 

^^^fljputSit ionai ". eff iciency, the ^roof of the general 
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'found in /4A; .Specif ically./r .-the -proof is both ;> • ■ .\ 
Vsimpler and more- rigorous.; Not only that/"^ but the_." 
particular .notational devices used in constructing ' ^ 
.the proof / lead ■ naturally * to the more efficientr* 
algorithm develdpled/ in -contrast to /4// where the.-- 
iiotational 'device^' edre suc^ t^at it is not so easy, 
to see thfe.!. (computationally) iOnpqrtant equivalence 
betwe^ the. cjiarairteristic root problem/ and the . . 
eigenvalue problem 'for a suitably constructed . , • 
matrix..- "Thus this paper sets out to ma3ce^ a 
contribution at : two levels : ■ f irstly / in respect of - 
computationai' off iqiency,. and secondly in respect 
of a further, consolidation of the theory -underlying^ 
the use of Z-transform' techniques for the-^adratic- 
linear optimization problem. 

■ ■• This paper' is .structured into two^'partsl' Part 
1 describes the basic approach ot- the Z^transf orm* 
. method'of - solution, . following /4/^but utilizing . 
different notational devices. :Part 2^ is where the 
real . contribution, of this paper liea, addressing 
the two . critical .issues arising from the, Z-transform^ 
approachr- viz. (i) the. -question of the existence of 
certain roots needed in the solution" procedure; and 
(ii) the question of an efficient computational 
routine for locating . those roots.- Thefvrelative 
efficiencies of ^ alternative computational approaches 
are then compared, -and^the paper concluded- 



; PART 1 - THE Z-^RANSFORM APPROACH 

Xn th-iTs part of the paper, the Z-transform 
approach to quadratic- linear optimization problems ■ 

■is described. Followiiig a^gener2tl statement of the 
problem, the' (first-order) conditions for a maximum 
aire , derived. . Some, restricting ^'(istationarity) 
eis sumptions axe. then introduced, and 'the Z— ^ansform 
technique used to show hcwd solution could be 
derived from the system of linear equations defining 

' the maximum. conditions* . . ■ ■ > \ 

:■ • ■ v x^-'.--. ■ . ., . ■• .■ 

the: general: PROBim • ^ - . 

. ' ■ - ■ * \ . 

: * . • The pirobliem under considjeratioii/ following ' ' • - 
Theil's /?/ canonical £orm, is the 'following:- * *' 




■ Find y ' which" . maximizes 'r 



^ - (X/y),. » arx<+-.b'y ^- bc^Ax + %y'By Jax'Cy 

T'- - ' 'i." : ' +'^y'c'xl^ (la) 



rLution is different, din certain'^'respects from that 



s.t. x « Ry + s. . 



(lb) 



That is, we have a "quadratic criterion in two sets 
of variables x amd y.- The set' of variab;Les y aire ■ 
the. deci&icn variables, which (it is assinned) ca^ 
l>e manipulated by the .xiecision-maker. The set pf. 
variables x 'are called >4c2te variables. These 
are affected by the decision, variables in accord- 
ance -with the- linear^ide relations given in " ■ 
equation (IbK The state V2L:iables. are also ■■ ■ 
affected- by the set of veuriables s/ which are* . ' 

- uncontr'olliible from the decision-maJcer* s viewpoint, 
and exogenously specified. 'The paxameters of the . 
problem- are the 'Vectors a. and b, and the matrices . 

■A,-B, Cji.and R. ' - * . * - 



•• The sets' of variables 
paftltioned as follows:- 



Xr y auad s are time- 



j ..DTxl 



■ 

^ That- is, for each of T periods in the planning 
. ; horizon, there are/ s' state, veuriables, ,S ekoqenous 
variables, and D decision variables,- so that x is 
of dimension STxl,. y is . "of - dimension- D^>cl, and s. 
is of dimension STxl^ , - 

'The vector and matrix, parameters a,b,A,B,C,R, 
f-are of- dimension^ which conform to x arid y.* ' That 
is, a is STxl, b is DT^l, A is .ST^l/ B is DTxDT, 
' C is STXDT amd R is ST>fDT. 'r 

The linear decision rule 'consists in. finding 

.the optimal /ir^i-period decisions, *that is yj, 

. * ■ -0 ^ ■ .; 

, when the planning horizon T as a- function oif * 

the exogenous variables,, the asterisk in^'^J' ^eing 

used to denote optimality^ Note here" that when . 
3C,-y..apd s are as defined in (2);, the exogenous 
variable s^ can be^ expressed as -a function of the 

' most recently observed state x^/. Tor' this reason, - 

writing a first-period linear, decision rule as a 
function of the exogenous variables s^^, 

•. is equivalent to a feedback ;cohtrc>i- law, 'Chow /2/) 
' ^ which the optimal decision in the t-th period^ 
• V*, is "written as a function of ^ the" most, recently 

iDbserved state. x^_^.-^ln this paper, therefore, 

^and^ contrary to Chow.'s /2, p. 17/ viewpoint, ^we 
take the pfacti,cal' view" that we Sire" dealing' 
essentially with > differences in computational • 
method when we compare matrix iteration, matrix - 
inversion and 2-trahsfora methods for.,,solving 
the problem stated. a - 



' i.e. 



where. 



a's:^ + ^s*As 



+ k'y > y'Ky 

■ ■■ / . - 



k. = a'R + b" *+ R.'As +-s'C' ■ 
■K. »-ij{R*AR + B + R'C + C';r} 



C4) 



(5a) 



(5b) 



(5c> 



Differentiating (4) with respect to y and 'setting 
.the result' equal, to zero, we get the formal, first 
order conditions, for a matximum,- thusr- . "•■ 



Ky*, 



" •We eure 'concerned, in this paper -only with ■ 
^' methods for solving^.the system of equations given 
by the f irst;order ^editions* The second-order 
conditions'' for a maximum are not discussed here, ' 
although it may -be ^remarked ' in' passing that 
sufficient, but- not necessary, second-order 

Jonditions are' that jcjbe negative-deiinite4'; See 
heii-./7/. for- a di^rfussion .of this issue. ' / 



ASSUMPTIONS 



We 



now ixvtxoduce 



the following 'assumpttions:- 



(i) 



(ii)- 



(iii) 



Civ) 



■ Assume that A, B and C are ^real^ ba^d 
block-<6.agonal matrices, with (time*-) ' * 
invariant band, and that A emd. B are . 
also symmetric. 

Assume tHat R is lower ba^d^lock- 
diagonal and*^ real, with C^ime^ -) 

invariant band.. i ' • 

* ■ . ■ • 

Assume that all- variables and parameters 
are real, if not-a;lready so specified. 



•Assume' ;that the parameters- A, B, C 
R have values such that the second- 
order conditions are satisfied. 



amd 



MAXIMOM CONDITIONS 



; ^ . The formal first: o^lder ^conditions for a^ 
:ma!xiamjv^a5r obtained by sidastituting foiMoc in (la)*' 
by making use of '(lb) r 'and then differentiating and'- 
setting the result equal. to zero (Theil /?/), as. 
follows:^ S\:ibstituting Clb) into (la), -we get:- 



Underlying assumptions (i) and (ii) is tKe. 

more -basic assimption that- cross-temporal effec-ts, . 

both in the criterion function and in the system 

dynamics, (the lineair side^relations) ■, are limited ' 

in- the number of periods .they s^ii; :>:Purthermore, * 
: th'e- assumption' is made that; th'ese .qross- temporal 

effects are" tirae-irtvariant,- so ''that' for. .any -given 
. cross-temporal span,. the relevant parameters are. 

the same, regardless of where in the planning . ' 

horizon the .'Span is 'actually taken. Considering ^ • 
' ; the. fact that we are letting T this would seem 

. t£> be a necessary assumption in most, cases* 

, , The -symmetry assuigtion for A anSr^ can be^ 
made- without loss :pf generaSlity sincy'we. ar-eJ dealing. 
;-^ith. quadratic f6rms\ The assumpti<^-:that fi^^is,'^- 
^ toiler band block-diagonal amounts to saying 'that 
current values" of the state variables are dependent* ' 
only'olf^evibus decisions, and"Trotr;?tt all -on future 
de<^isions, which- wb^uld seem to be a necessary 
assumption in ^irtost*P^blem- situations. 



C» (y). 



a' (Ry + 
.+ ijy'By 



s) + b 



+ s)^ ^ 



ERIC 



s) •A(Ry .+ s) 
^'C, CRy +.S) 
(3) 




y The assumption that^ll parameters aore real 
needs no justification. It would be difficult t;o 
give a practical- -interpre'tatipn of complex . ' 
parameters in most applications. 



Following on assumptions (i) -and (ii) / we 
have the result that K must necessarily be band . 
block-diagonal r symmetric and real. This res^^t • 
can be quite easily proved, though with sorae tedium 
using the; defining relation (5c) . 

Taking the relevant cross- temporal 'span* of 
the system.to be 'd' periods, therefore (this may' 
.Ijena system lead' tiae; for example, , in an 
eaudatioxial ^stem, the' liaad time between initial ^ 
exiroliaent and final gradliation may be considered to- 
be (at most) six years, or periods^ in which case 
d . - 6) and with the symbol {• ) denoting trans- ' 
,posi^on, we may write K as. follows:-,^ - - ' ■. 



t with the" infinite-band portion of (7) . 
'defined by .the following 



^(z)'is. 



k ■ 



■d-l. . . d- . d-1 ■ 0 



0 - d'l d d-l 



K 



d-l 



d-l 



DTxOT 



K 



"d-l 



(7) 



This is merely- a general characterrsation of a 
band block-diagonal, real, symmetric ma&ix- When 
T «, the infinite-bapd portion has "width" 2d+l, 
made up of the 2d+l block elements K^, , 

K,/ K - - , , Note that-; in this character- 

^ a a— X ti \ . ^ 

isatiox^ we must .hiavB that is symmetric , ' which 

it is, from (5c)-: Note also that- each bldck 
elenent has dimension , and can ^e* .chained 
from (5c) by. first time-partitioning ff/'iB,* . and 
R/ into block elements conforming tq ^Assumptions^ ' 
(i) .and (ii)r and then czurrying out ^e indicated 
operations.- 'The result C'7) which then follows 
will, form the basis for the method of solution, to - 
which we are headed. 



METHOD OF SOLCn'lON 



Substituting (7). into. (6) ,- letting. T «, 
and' taking the Z-transform^ of tlife result, we get: 



2d-l. 



•. Z 'Q*(z)i* + z'^(2)Z^J^).:=«. ZCk. ) 
i=0 - 



C8) 



where 2 ( . ) ^ es tShe . transform o f a (ve c tor , . - ■ ' 
in this -case) s^^tttoce of ! variables; 5^' (z.)-,' • * ■ 

i "» -0/ 2l3-*L' sure the- polynomial matrices that 

premultiply ^J, , ijd-l' ^^^^^ - ' ■ 

. obtained upon t^Jcing the Z-transform, and P (zV- is''^ . 

O I characteristu: pohp^iomiql (matrix) assoqiated 



d^l' 
■i»0 



. i-0 



1^ 



(9) 



Note tjiat for any arbitral^ value of the 
variable z- (9) , is an equation in the. (2d+l)D 
variables vj, and Z{^). ' 

• Now multiplying -,(3) ^through by F (z).; tiie 
adjoint of F(z) ;^ and. making use of the *f act that 
for any . general square matrix A, AA |a[ If. - where A 
denotes the adjoint of A, '|a| denotes the ^etermin- 
ajit of A^ and I the identity matrix, we get:-- 

. :2d-l.'- . . ' ' 

. S^Zi^ + z I^'I2(ZJ) =-F.2(k^) ^ (10) 

where the; z arguments denoting the dependence o£-; 
the polynomial . matr ifces F , T-, ^Q^ , - . . • Q2d-1' 
.on z, have been, dropped for notational .simplicity. 

■ Now,. rif 'we choose z; within, the unit .circle -. - 
su<A/*that\'lF.(z} I =' 0,' .the ■■term;in**Z (^') . drops. -but;.. 

and,' if f mrther, the ■ sequence {k^ , . t = 0,1,2/ •.'•}' 
is bounded above, .then Z (k^) is finite.^ . We wpujLd 
tlius have '2ui/eguat ion in only the 2dD variables 
Y^' -^i' " ' ' .i2d-l' ^-^^ ^ ^- ^® .'" ' ■ i 

definition a root of -the characteristic polynomial 
|f(z) I*. Furthermore, we^know from the result in 
Linear Algebra /3, p, . 61/ that T^) has rank unity 
for any root of \fXz) Consequently/ substitution 

in (10). of any z , say, within the unit circle, 
m 

which is a root of jF(z) j / yields exactSf^ one 
lineeirly independent equation in the 2dD variables 
Zq/ ij/ f iJd-l' taking a single 

Denoting such a row by 



non-zero row from r(z ) 
m 

5^(lxD), we have on substitution of a- root into 

(loy, one^ linearly '.independent equation defined o.,^ 
thusrr , ^ . ■ ■ 



2d-l 

f^' { r QAz^)^} = 2„Z()c ) 
m . 1 m ■■ m --t 



(11) 



i=0 



In order to solve for the 2dD variables . 
- f.yJd-l' J^sed exactly ^dp. linearly 

independent equations* "We^^m^y obtain dD linearly 
' independent ■. equations . from the original conditions 
for a local ' maximum . * Observe the re that . the , first 
dp. equations contain only the 2dD variables 

ij, ..^ / Y^^^i' The ^remaining dD linearly " 

independent equations may be obtained, from equatiop 
, (11), by substituting dD- roots 2^ 'Satisfying^'f2^|.<l< 

■m .--l,.2-, ,;''dD. -It is obvious that this pro- 
cedure can 'only work if there are exactly dD roots- ' 

■•of. |f(z)| 'lying within the' unit -circle of the _ . ■ - 
complex plane. ^ Furthermore, the pbmputationatl ■ ■ ■ 
ef ficiency ^of this procediire' depends critically 



.".'upon the efficiency with which 'the roots z^, 

• ^ m_-« l^J^ ... , dD caul, Se* found. ■ TheT" rest of this 

* "paper addresses, itself ■ to these^ two questions. . 



. ■Note -in:passing, thfl^^the occurence' of complex' 
roots' poses Tio insuperable problems. ' See "5/ B/. 
for methods of handling" complex roots. ' 



■ ' . ?I£2lr. proof follows, ^easily from theL 

definition. of a symmetrical polynomial, and is 
therefore omitted. " 



■ ■■ -PART 2 - A- PROOF OF EXISTENCE, AND. 

A METHOD FOR rimiNG THE HOOTS ' ' " , 

In this part o^" 'the, paper, firstly we prove, 
the existence'of exactly dD roots of .[FCz) [ lyi«g.*. 

^.wjthin the xiait circle of the complex plaiie, using 
^a different and , more .rigorous approach 'than' Hay ' 
atnd Holt /4/. -Seconaiy , ' we describe an efficient 
method for obtain ihg the dD roots of |f.(2) | lying 
within -^e unit circle. Thirdly,, a-brief , * , 

^ comparison is" made of. .the relative' eff^iciencies 

"jbf the alteri^tive' approaches.- 

.• ■': ■ . ' " ■ • 

EXISTENCE PROO F '■'""■: •' ■ . , - " ,.*'"■'" 

:: : : V.-. .V : .."^ '■ ' 

r ' To-.prove the ^existence of dD" roots of If{z)|. 

'• ■lying within., the unit circle of the complex plane, ■ " 
the approach is--as follows: The co'ncept of a 
syrTnetricaJr. polynomial (Definition 1> is introduced, 
and it . is proved that symmetrical polynomials have 

* an' even- nuiciber of roots, ^ half of which lie within . 
the- unit circle. of the -compilex. plane. (Theorem 1). ■ 
It is--then' shown that '|fCz) is a symmetrical 
polynomial .of degree^ . 2dD (Theorem 2), from -which 

•^the requiried result immediately follows - (Theorem-- 3) . 
To prov€f 1Aat .|FC;2} ;]*is.; sr^^ it is. necessary 

.,to introducti the'concept of the r'everse of a 
polynomial . (Def ihit ion 2 ) and to . _ prove f irs t ,. 
several lemnafcaC involving this, and the /concept of ■ ; 

'symmetrical polyn93iialis . V ^ - ^ 

\ - ^ DEFiNiTiowl r ' \ symrietncal'^ 
-polynomials of- even ;degr«e' .2n , ,^ay , 

that a ' 



polynomial- is. a 
2n'' ■ r ■ ' 
■ a X such 
,rrO. r 



f (X}. 



*2n-r 
»"3 + 2x 



■ 9 ■ 



4x^ 



2x-^ 



3x 



Example : - ^ 

■4' - 



is symmetrical. 



. DEFINITION 2 : The reverse of an n-degree 

polynomial , ^r^" defined tp be'tWe n-degree 

^ r' \*"' 

polynomial ^Iq -^j^^^^-- denote the reverse 

of 'a polynomial. Example:- 

R-Ta'-f- 2x -K 43C^} =-.4~+. 3x^ ■ ■ " ^ - 

■/ / ' ' ^ ■ ... ^ - ; . • \^ 

!verse .of an^n-degi^ee- polynomial f (x). ma^ 
alternatively- as ■ follow^:- ' ■ "^^ 



be def 



"RCf.^x)'} = x^f (i) 

. . ■ ♦ .X - 



. THEOREM j. : A - symetricot polynomial' of degree 
'2k pcBse$3e$ exactly n roots, x*, ^ = -2^ ... j n, 

sane pos'szbly repeated, Satisfying' .\x .\<1, 

' » ■ 

i = 1, ... i n; provided only that there is 'no root 
X such- that \x\ = i. " • ■ ' ' 

Proof : For any (in general complex) number x, 
either Jx|<l or^!xI>l, if |x|j^l./ ■ . • 



Furthermore, |x[>l H<1. 

x^ 



>lx - 



0. More- 



* Now, by Lemma.l, tf (x) =^ 0 <=> f F) 

over, being of degree lin, f(x) must have exactly 2n- 
roots ' (some possibly repeated).. • ^ 



By symmetry, therefore, f (x) must have exactly 
n roots satisfying [xHl and .exactly n satisfying 
|x|>l (some possibly repeated) ;. provided only that 
there is no root x' such -that Jx| =«1.' QED. 

LEMMA 2 ; The sicrn of synmetricat pctynamials' 
of the sane degree -is also s'yrmetrica^, , • " - " 

' . ' Proof : ^The^proof is obvious '.and therefore 
omitted. • ■ . . ' . • ■ " - V . 

^ LEMMA 3: The swn of OK even-'degree polynomial 
: and its reverse is'^a synmetrical polynomial. That 
isj f(x}-f-'R{f(x)}i3 symmetrical if J (x) is of 
even degree,' . ' - ■ . ' 

Proof V : The' proof follows easily from the 
definitions of a symmetrical polynomial, and '■bie - 
, reverse ^.of-, a polyi?omial, and is therefore omitted. 

^*: L£MMA 4 : . The reverse' of a product of. 
polyiv2mial^~\i^ the "product of reverses of the' 
individual poh^naqsi^ls making uo the product. That 

■ ■ ' " : " ■'^ ' -.. . ■ \ \ 
Proof: The" proof, is by induction^ It i^.i^. ^. * 

shown first- of all that if the fesult^holdi -^ru'e* -'/-^ 

for the product, of p. (p*> *2}' polynomials, .tiat- 'it ' 

must; also 'hold for the. product of p+1* polynomials. /" 

" ' . ' . , * 

■ , Assume ^ , • >• ^ ■ - 

" • ' - ' ' ■ -.■-'..-■■^ ^ 

•£.}•"= R{f,}-R(f,}. ... ■.R{f }. 



Theh^ 



- . ■" he equivalence of the two alternative . ^. ^ 
".s definitions maybe easily proved* Example:- 

. .. , . / Rfi +^2x +i4x5} » x^(3 'f ^ + ^) = 4 +^2x +-3x^. 
• . - . . X x^ 

' ■'''*.**'.■•■ 

. * as before. •"•!.,''•*''" ^ ^ * 

LEMttA' 1 = 'Yf 'x^'is d^root of^ the 'symQincdj} •* . -A 
• . ♦ polynomial ffxJy.- 'then its r^dwocat^ is also- a - ' - 
" ^ V root. -. That is, if f(x) syrmetrical, f(xy--' . 
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: f ^-f^ } = f ^^.-f ^} • R{^ ^ } • 



R{f;}-R{f,}-R{f,}- -Rfft^i,}.' 



^(12) 



(13) 



That-^is^."the result must then, also hold ,fOr ^1-: ./ 
polynomials.." . .'."^ > •■,.' •'. ^'^ '.: V...*V 



.Secondly, 'the result is proved for two- 
polyn"omials. Thajt is', we wish tb'*sh6W 



RCf^-f^y - RCf^l-Rffj}'.. RecaU f irstsfroi the 
J * definition/ th*t for an n-degree polynomial fM , 
* R{f^Cx)} -.xVtj)' Now "suppose that f^;Cx). is bf^^ 



degree ji^and that J^^^^ degree nj- T2ien; 

• • ' . ■ ■ > . ' ■ " c 

' R{f , }-R{f 2I - x'^'^^f ^ Ci) ; f 2 (jV, ' ; 
by definition of reverse. Now, let 



(14). 



Thus 



n+m - /1» 



byyiefinition of reverse; that is 
by '(15K Hence, by (14) 

^- ;.Rtf >f2};:ixf,>R{f2J-^ ; 



LEMMa jte even-degree poZyrwmiaZ^hich 
* ita CM rever^ is a syrmetrnr'cat potifr^^ That 
is, f(x) - H{f(x)^ fCx) is symetricdl^ ■ 

/ ■ ' _ - . • . * ' ' ■ 

Proof : The proof follow^ jsasily from tne 
definitiion of a symmetrical polynomial, .and is 
therefore omitte^. 

' THEOREM 2 :, ^Tfte detcmiruxnt of F(z)y where 
■ , FCz) is the poZynamiajL mcri^rix as defined in 
— '^quatCon ( 9 )y if^^a^syrmetricat potynamiaZ of 
degree 2dD* 



Proof: 




a polynomial matrix whose 



i5-th elCTent Is a polyr.omial of .{even) degree 2d 
. given by equation (9)/ elaborated her:e for conven- 
ience. • - 'r 



d-1- ' _r rv" ■ > . ■ d"+k 

i, j^'l, , Vf^ 



of f»^^(z); that is f .^(2>-^ Rff^^n(z)}, i,j. 



.226/ 



Now by definition-of ■ a ■detenainant /6, i»p."225. 



(17) . 



-where a, (1) , 

k" 



A*" 



C19) 



r <5'j^ (0) is the k-th permutation of 



"^ (15) „■ . ..the'. integers-.!, ,.;r ,;p where the sum is taken over 
. .* ; all such permutations. The symbol (sgii) ^ denotes 



•fl if the^ integers o^il) , 



(D) can be con- 



verted into 1/ , D by an e^en number of inter- 



changes'; ptherwise, fsgnh 



-1. 



' ' ■ ' Note , that every term' in the summation consisJ:s 
. of a -product of, D .polynomials'" each of degree 2d. 

Cons equently | F (z ) | \is ■ a ' polynomial of ( even) degree 
, 'SdD, provided of . course that is non-null, which 

^ we- may assume without loss of generality. 

Furthermore , every term in the summation ii; 
either symmetrical, or . has 'its revers e in the 
• ^summation. To prove this assertion, consider the 
k-th tefm.in the summation 1 



(20) 



Since Jfj^(z5 - R{f^j(z)}, i,l. « i, ... yd" and in - 

view, of Lemma 4/-' the- reverse of the '^^^ term can 
be obtained quite simply by reversing the subscripts 
"on the polynomials in'the product and keeping the. 
' sign,' to get:- ^ . . 

Now let <yj^..(l)/ ^y.^ C2)r r Cy.t (D) lae the 

permutation of 1,2, . / . D> defined by ^,the first ^" 
5U}3Scripts orv rearranging (21) so tha^ the second 
'Subscripts- are in t..s order 1/2, ..... ; D. But 
<y. , (1) , a. , (2), .... , cr , (D) is clearly mother 
• ^ ' * , '• ■ f , ' 

permutation of 1,2,. -.7, D, vith (sgn)^^- (sgn)j^r; arid so^ 



-where (- ^'^j denotes ^e ij-th element of/ the mfeitrdx^ 
■ i». brackets." - , > ^ ' ' ' " 

. ■ ■ ..^v--. ■ -X - ^ - ■ ' 

^ Haking .-use of - the.' fact .. that ^ K^. - ' , and ■ .that ' 
' ji " ^Vij ' ^ - 0,l>i2r VI. \ \ d-1, by. 'the- 

definition of transposition, we have # 

. •' ■ • d • ' ' - k - d-l ' ■ d+k 




(18) 



i,t*""t is we have shown that f . . Cz) is the reverse 



ERIC • 



.must- be -a term in the summation^ If it is identical 
y to ^e k-th term,* then by Lemma 5 we have that the 
.. 3c-th:term i» ^mmetrical. 'tf it is not identical 
to the' k-thiterm*, it' is by construction, "the reverse - * 
of the . k-th term. . Hence we have ^ the result that 
every term in the summation in (20) is either' 
symmetric^ or has its .reverse /in the summation, —f- 

• Therefore/by Lemma's 2 and .3 / we'have that: 

the summation of 2dD-degree. EPiyr^o^ials defining 
|f(z)'| is symmetrical,, and of degree; i23D. QED. 

' THK)^M_3 ; l^i's^l possesses exactly dD.s^ootsy^ 
■ z^y i = T~..7'\^ 'dDy- some -po'ssihty: repeg:tedy sqty^- 

. there-exists no'rch't^- siich that - I*^ ' • . . 



Proof ; By Theorem 2^ we have that iPCz)] is 
a synnnetrical polynomial of de^ee 26D, which is 
■ even. . . 

By Theorem 1, therefore, |f(2)| must possess 
exactly dD roots, Zy i - Ir.... , dD, sane 

possibly repJeated, satisfying I2^|<1; provided- 
only that there exists no root z such .that ]z\ «.l. 

■ . . ' > QED. 

nroiNS Tas roots , ■ 
*^ 

The ccoputational efficiency of the 
2-*OT^^sform method of solution depends , critically 
upon yie method used to find the. roots of |r(2) [. 

• Hay and Holt^/4, p. 242/,report the use of gradient 
methods. The efficiency of this approach is not 
discussed in that paper, but the authors seemNto 
suggest /op.cit. , p. 257/^ thaf matrix Iteration or ' 
matrix inversion are ^likely to be superior 

.approaches . . The method that is suggested in 
this paper^ seems able to make the Z-transform 

method conputationally superior to either matrix 
iteration or matrix inversion, and is -developed in 
this section. . ^ 
\ ' • ^ > . .- ■ _ ^. 

The method. £s as follows: To find the rpots 
of [PCz) |, first form the 2dDx2dD matrix. 



^2 . . "2d 

■I 



K - 



-I 



where the time-partition k •« Ik^, JS^' j 

been, used-- Note that (23) defines a set. of 
difference equaPions of order 2d. F (2) is the 
ahzpactei^istia polynaniat matr^J: obtained from 
taking the Z-txansf'orm of -this setr of di^erence • 
equations. To pr9vie the theorem, (23) will be 
^writ5:en'as an equivalent /trst-order differ ence"* ' 
e<5uation, vrtiose characteristic polynomial is " " • 
(zl + <). The roots of the c?m'a<?t52*£at^- e(7ui2t£^7?i 
obtained in either case must be identical, 'and- in 
the latter case the roots of the characteristic 
equation aure the eigenvalues of the matrix K,'by 
definition.^ • ^ j: ■ ^ 

-1 ' 

Now, premulf*plying (23.) by-K^ , we get:-; 



. 2d-l 
Z < 
i»0 



Now, let 



2d^iZi^i ^+2d " ^0 - iSt^^t^o/v^:. 

• - . (24) 



rCi) 



and define 



^+2dTi' ^ " ^'2' ' 

< t = 0/1,2, .... . 



(25) 



t • 



~t 



,(2d) 



i t ^ , 1, 2 , ... 



<26) 



i ^' '.: 

■ ■* ' ^\ . ' . " ' 
The 2d-order difference 'fecjuation in (24) 'can there- - 
fore be equivalently written as the follovdng first- 
order difference equation:- 



i- '^^^ Vl - V ^ = 0,1,2, ... 



(27) 



where « 



Kq K^' , i"l , ... , d 



where we have let 



and compute its. 2dD'eigerwalue3 nsintji one of the 
several available pom^ter programmes /9/^ to do 
^is^. The eigenvalues of . this matrix are . the , 
Sroojls, of^ fF(2)[. Nbte that tliis procediire 

■'■ ^ * -1 • ■. ^ - 

rejjuires that exists. We therefore wish : to , 

prove the following, theorem, ' . ' 

.\ THEQRSW 4 r -The TOOts of \F(z) \ are Identical 
to the eigemaZvea of <, provided that exiate. 

Proof Substituting. (7) into (6) and letting 
T as before, we have from 'the infinite-band-, 
portion:- .' • 

t * iio Vi4^*i ■ ^ : . ■ 

V \\\ " '"^^^^^d' 0,1,2, .\-: ■ . : ^(23). 



^-Hd 



0 : 



t = 0,1,2, 



(23) 



2dDScl 



The characteristic' equatibij of this system 



1 21 -he 



(29) 



the solution to.* which is given by the eigenvalues, 
of ic, by def inition. But, .from (23) the character- 
istic equation* of the seune system is; given by 

. \t(z)\ = 0' ' ; .■ ' ' : ' 

Hence, by a _ fundamental proper^ "of 'linear dynamic- . 



systpins, we have tha^ the Qigenvalues of < must be 
identical to the\rooes»o£ |f<z) | '. QED. 

CCMPftRISQN OF APPROACHES - ' 

In the 2-transforra- method suggested, the 
number *5bf arithmetic operations required is 
dominated by. two steps ^ viz. ^ locating the eigen- 
values of a 2dDx2dD matrix^ and ii^^rting a ^dDx2dD 
xaatrix. ^ Each of^ .these steps' require in the order 
of (2dD> 3 'arithmetic operations to be carried out, 
say k<2dD)^ altogether, where k is an appropriate 
■ factor.'^;-;, • • , ' - ' - ;. 

Matrix" iteration methods (e.g. Chow /2/) assume 
a finite horizon We c&n write T ^ nd, where n 
is- a'^'ntznber chosen so that T approximates infinity-/ 
insofar as the convergence of the first-period 
solution is ' affected. . (In practice this would seem 
to require n-f 4/ if the ^^-e suits in Theil ,/7^ p, .167/ 
are ^any indication}^. The matrix .iteration method 
"*rec[Uire4^ number of arithmetic operations 
^dominated at each iteration by the multiplication 
of two (d+l)D X (d+l)D matrices. CThis assumes 
that in the case of lagged wieibles , the state 
vector is augmented, by these lagge<5 variables so' \^ 
as to Iretain the first-order, system dynamics^ , . . 
2^ssumed. by Chow /2/, and that the state vector is 
of order not .l.ess .than the decision.>vtetor} . This 
multiplication, requires in^ ^e ord^-of 2(d+l).^D^- 
' arithmetic operations r and needs to be ^zirried out' 
nd times altogether. Matrix, iteration therefore 
requires in. the order of 2nd (d+l) Operations * 
This number of operations would therefore exceed 
that for the method suggested rdughly whoever 
* 2nd (d+l > 8kd^. Assuming n - 4^ this is always 
true if k < 6, whicft will aliabst certainly be ^the 



Matrix inversion (e^g. ■ Theil T^/) would 
proceed by a tt e ro pting to invert the l2u:ge', matrix 
In C7y, 'for T l2U^e enough to apprQScima^- infinity 
and ensu3;e* convergence/ of ^ the lirst-period' 
solution*. "The "number ■ of arithmetic operations *; 
required to -perform ^hi^ inversion is in the 
order of (DT)-^. Writing ^ ".nd, as before, this 
is in" the order of (ndD) ^ operations. 

^.Assuming n 4> this nirtfther would' exceed 
k(2dD}^, the number required for the method 
suggested, whenever Ic < 8, which again will almost 
certainly be the case, 

Tliese'are very rough calculations.^ b^ they,' 
indicate that the ^-transform method, is- likely to ^ 
be- cccnputationaJ.ly superior to either matrix 
iteration /Ir p- 78-et.seq. , 2/ or matrix inversion 
/?/. This is contrary, to what Kay and Holt /4/ 
suggest, but is based .on a^ different method of 
calculating the' chzuracteristic -roots ,> than the^ 
gradient^ methods to \rtiich they alliide /op.cit., 
p..242/ .'^ . . ' V : . ■ . 



CONCIUSION ^ . • • 

This "paper has 'sketched an ^algorithm b£ised oh 
2-transform' methods, for the n;znerical solution of 
the optimization problem, in. which the ^objective 
f sanction is quadratic, and the^ constraints are. 
linear difference- equations, . It ifS'-shown*' that .the 



method suggested is likely .to be more efficient than 
either matrix inversion or xnatri?c iteration. 

Also included *in this paper is a proof of a 
' theorem guaranteeing the existehce of the requisite 
number of roots of a "chatracteristic polynomial 
satisfying conditions for the successful application 
of the Z-trahsform method to the problem.- A recent 
proof by Kay amd Holt /4/ u»es' a different approach 
'emd is perhaps not as rigorous. • . ' * . / 

The not too restrictive assumptions- on which 
these developments have been based are that* the span 
of cross-tsemporal-, effects be finite # ai^ the system 
parameters, be 'tiiier invariant. The other assimiption 
that all parameters be real needs to be made in any 
case. A nusiber. of ancillary issues and extensions 
discussed in Hayvand Holt /4, p. 249/ are not 
discussed here, specifically the treateent oi 
■ complex roots, zero 'roots, time-varying pairametersr 
and tlme-discbunting of the objective function. 
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'FOOTNOTES ' '-^ ^ ' - 

^ ■ For a scalzur. sequence, (a^j^ - 0 ,1, 2 , ... • ')\ the . 
2-tr2uisf orm of this -sequence is defined 2ts 

't' 



^(a ) " J-^ "^^ where z is a complex variable/ 

t .. t^O X , 

' * - . ■ - 

The Z-transforra of a vector sequence is defined as 
the vector who^e elements are thev Z- trans forms of 
the "indivi<iual- elements of the .original vector. See . 
Frazer et al /3/. for a discussion of the algebra of 
Z-transforms.. ' ^ ^ ; ' t ' 

^ More precisely, 2(1$^) " z^ is -finite 

if the term' k^' grows at a^ slower rate than the 'terra 
z.^ declines 

^ In /S/" on which this section £s based, , the 
word "symmetric" weis used. The term "symmetrical" 
is used here in a narrowly defined * sense, without 
regard to any other meaning which might be attached 
>to the vrord in the mathematics literature. 

?** It should be pointed out that .the existence q^^^ 
a root z such that |z| - L is 2t- very unlikely h^£:^7'" 
line case, almost' impossible to- achicve^^ numeric%i' 
* computations. 



5/ In the case, where C23) were aj|^^^^|^£erence 
equation, the matrix < would be lap^RgW^'^e . *r 
adjacency matrix^ and Th^rem. 4y^ui^^fc^ . known 
result- I am indebted^ to D. Beckles -^SSfl^oinlSLng 
this out. The -extension , to' vector ^difte?^ce , ' 
equations implicit in Theor^em 4, thougt. satiewhat 
straightforward-/ I 'haven'^'^eea^.able-^ find in the- - 
literature* ' - . ' ' ' ^ ^ ^ . 

-■ ' '■ •■ • • ■. ." ■' ■ 

^ ^ First devjeloped in" /8>/\^C^^ubli9he4) ^ ^ . ■ ^ . - 

;.' It , is ->^ll known that^^arix . fhversion requires 

in, the order of n*^^ operatacj^^ for a matrix of order 
n. See /6.'-p.v69/. - The assertion tHa.^^ the order. 

of n operations ai^-re^ired *to locate theP^eigen-. 
values <3f a ^ener^ISaLtrix'of ofdex-;^/is biased; cjn 
an jaioalysis' of .algorithms "found in /9, contributions 
11/12,, ir/lV- Also, see /6, Ch. 10/. , 1 ' 



THE STRUCTURE AND ;SOLUTIO^f TECHNIQUES OF 
THE PROJECT iNDEPENDENCE • EVALUATION SYSTEM 

• FREDERIC H. MURPHr 
• ' FEDERAL ENERGY ADMINISTRATION- 



' 'INTRODUCTION ■ • - ' 

. • ^ ■ . ■ ' » ■ * • . . . . . ♦ ■ ' , ■ _ . 

The Fro^'ect Independence Evaluation. System^ (PIEiS) forecasts the state af the energy economy in 'selected 
future years {1980, 1985 and" 1990) and reflects the^ impacts o? a range of potential Federal policies on 
the prices paid for energy commodities, on the Jev^l pf-demands for these commodities, and on the. leveV 
of imports of oil.' The methodology used assumes thtat the role. of igovemment is to establish policies - 

. . allowing participants in the economy^to act in their own self-interest Within the constraints imposed by. 
these policies. Th^ approach taken^ fs to construct models for. the different components of the energy 
system and'then integrate, the submodels or .the outputs* of the submodels into a forecast^ This modulari- 

\>zation allows for the ongoing improvement of the various segments" of PIES without having-to alter the 
Entire system- - ' * , ^ ^ 



OVERVIEW 



There is ar set of -supply model s for each of the major raw materials, coal, oil an<i gas. They are built - 

•: . ■ ■■ ■ i -■ '~ ■ .... . ' '■ .■ 

. to simulate the response of the industry ^producing the raw^ material to* increases and' decreases in prices 

. * and are used to- construct supply curves. Next, miniature: models of refineries and electric ut.il ities: 
• • - - ^ ' ■ " * ..... . ' . ' ■ ■ ^ ■ - . - "■ ■ 

* transforTn'raH materials' into consumable forms of energy. Estimates of the "production capabilities of 

emerging technologies are. added as Wel^ tli^ products conisumed within the system are six petroleum _ / ^ 

productss gasoline, disttHiate^ residual, jet fuel, liqirid petroleum gases and other products from crude 

oil (tuBes,; waxes,, etc.) V and four other products, natural gas, electricity, bituminous coal and 

metal Turgical coal . A 1 arge data . base and set of econometric model s are used to construct a demand 

" mod^l which estimates how the^demand for each final product varies ais the price\<)f that product and the - 

.prices or 'Other products changdT^As an example of how the price of one product impacts the demand for 

andtber, natural can -be replaced* by distillate for. many industrial useson approximately a BTU for - 
" ' r ■ ■ . . . ■ - ■ ■ : ■ ^ • ■ - ■ • . . — : ' ■ ■ ■ ■ ■ , . 

.rBTU':basis and vice versa*.' Therefore, if the per BTU price of one fuel gets out^of line -with- the. other „ 
- • ^ - ■ ■ ■ • ■/■ \ ■ . . ■■■ ■ ■■ V ■ ; ■' ■ " ^. • ^ • ■ . . 

j ^and both are available, the lower-cost fuel, fs \substituted for- the higher-cost fuel Sijice natural, gas 



> ! and distillate ar^ not perfect substitutes, n»re^ becomes economic and ^o'ccurs* in t^^^ 

.. as. the pricesj^of *thg ^;vwjggm ^dities . continue.:to .diverge. The demand function-is a Tog-linear approxima- 
tion to a set :o£,^Sb^^^pecif\csdemandTO^ - .'i^ : - - T. 
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: V where q'^ 'ts" the ^q^^^ product j demanded, sector k vfhen fue^^sell at the retail prices p . The 

sectors are household, corimercial , raw material . industrial and tfahsportation.' When i=j, b is refers. 
' red to as the own elasticity for product i and is negative; If i?«j, then b^^- is a cross elasticity and" 
. " Ndn our case is positively signed. With all other prices constant, an x percent. change in p^.. leads -to an 
. . xb^^- percent change in the demand for product j . . A full discussion is contained in (2), describing-. . ..^ 
each SfeOTeat of the demand, model in detail. 

" Oemands'are forecasted for a ^arg^^slate o1;^produGts than is available f rom. the supply str;ucture. 

Al so, the supply prices are wholesale .as opposed to retail rprices. To evaluate the ^jemands for the ten 
H supplied prodijcts at the supply (wholesale) prices/the following steps are taken. First,\^ each; demand 
product, is associated with one of the supply products, e.g., petroleum coke with other, petro^um 
prDducts. Markups appropriate for going from wholesale to^retail prices in the given sector are then 
. added to the wholesale prices and the- demand equations for each fuel are evaluated. The resultant 
■ quantities are then aggregated across the appropriate demand products and across sectors- to determine 
the 'demand for. the supply product at the.vrholesale price. ; .. " 

The .various components of the system are .tied together by a transportation netwa^K that moves raw 
materials or products from where they are produced to where they are consumed or where they are used to 
produce other energy products. 'The flows wi than the system.Ve shown ifT^Fi gure- 1 . ' ; 



^InQj 



+ z 



1=1 ij. 



Inp for j = 1 , ,10. 



FIGURE I 



Flow of Materials 
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The supply functions; the. demand functTQn\^^.^ W^ conversiorPactivitie^ are combined by the PIES 
• integrating system. The output ;of the integrating system is a gene/al equnibrium^ solution (Figur^v^ 
of the nwthematical represenprtf^n of the energy economy,, i.e. , a set of bdTanced%u^l ies and demands 
as fifel^l^as market-clearing prices for each fuel- is provided. Foniially/stated, the problem ,is to find 
a vector o-^ prices p such that the vector of demands D(p) -.^S(pj the vecto^ of supplies at p. 



^ Integra ting Model, Framework 
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REGIONAL STRUCTURE ^ " . ^ ■ - / - , . 

Each raw material has a regional structure in the model that represents the unique characteristij^ of 
its resource- base.' There are also specific regional definitions for conversion and demand activities- 
The purpose of all of the' regional jdetail within the model is not to provid^'^esults for regional 
analyses but to provide better national figures. Westei^h low sulfur coal would -implicitly be used in . 
New. England, for example, if transportation cost differentials were ignored. Also, more nuclear plants 



would be bii^fel^ if a national average transportation cost, were used for coa>. 



There are twelve coal regies which are chosen so that each is relat.ively 'compact and contains only a 
few coaj categories of bituminous coal , ^me regions also contain TOtalTurgical coal. Coal is 
separated into five BTU 'categories, and each of the bituminous'ca^egorTes is divi<led into three sulfur 
types^ 'high, medium^^ai^low suKur. The coal regions are shown in Figure 3^ Transportation is a ^ 
substantial part of the. costs in using coal- Within the jnodel , the more compact the coal region, the 
better theestimate of rtransportation costs from the coal region to the utility or demand regions. 



As a consequence, even though some regions such as Central and ' Southern Appalachi a contain the same/ 



categories of coaT» they are modeled as two separate regions to have a better estimate^of tranipbrta^x^^ 
costs. To further, refine th^e corts of shipping, a' t^^^^ 1s used. : Ccial' moves from the " 

coal, region to a .col>ctibn of transshipment nodesr c^.ties sucfi as Cincihnatt..i New'd^^nt^ 
Instead- of shipping the coal; to Jhe' demand regions from 'the .trans shipment po^ts. it i's shipped to a 



'Selected s«t of cities in the demand, region ^ith the demand -in each city' a fix^^raction of the demand 
. region n^eds.^ ' _ - " * f ' T . ^ > * / " 



FIGURE 3 
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There are 12 oiland \2 gas^regions based on National Petroleum' Council (NPC) regions and special ■ i 
"Alaskan Regions. The refinery regions are Petroleum Allocation for Defense Districts "or PADD's. the 

crude oil (condensates, etc.> from gas regions) are moved into the^refinery regions from the oil and gas 
' regions by pipeline 0^ tanker. And the six product groups are moved from the* refinery regions to either 

the utilit)^or detpiand regions. 

For ease in modeling, the utility arid demand regions are the same. They are FEA regions. . Unlike the - 
regions for supplying other forms of energy, a utility region.may serve only the corresponding demand 
region except for the shipment of hydroelectric power from the Northwest to Californfa-. This greatly 
simplifies the calculating af the average cost of ^electricity, that is, the price 6f electricity to 
consumers. . > .. ' * m 



SUPPLY ACTIVITIES .! ' ' " ' 'V- ■ ; . , 

■•"^^^••''t'^a^'*'?'^^^ used by economi sts is to}jest'^^ a^i^i^tf'fijnctjon. ^ 

>*v.wfth6ut se'ribus Tegai^^^tp "the resour^e-^teTe' (see, for example. .is inappVpria^^^ 

' the most important factor, affecting the supply of fuels is the character ancf extjn^^^^ 
Rather -than using historical time ser,ies data- and statistical te(^iques to directly predict^ future raw ' 
*. material- and product availability, operations researcl>-*a^sed ppi^^ are" built, to simulate' the., '^^ 

' actual productt^^ ^ . : . 

■" ; T^fi^"?^^-^^^^^ ara used to construct supply curves :that are step-func€ion yproximations to cpn^inous 
'- functions. .For example, each step of- the coa.l supploT'Curve "^r each region 'represents the annual rate • 

^^of production- from a specific mine type within two mine'classe^: 'surface and deep. In Figure 4 the 
• lowest-cast steps on. the coal supply curves are .associated with existing mines or mines that ^re about 

to be. opened. Here the capital is sunk or mostly sunk. and -the mines t^ill be operated long as the- 

marginarrevenue is at least equal to the operating cost. The higher-cost steps ensure the- capital 
_ recovery necessary for ^opening a.new mine.- There- are-supply curves for each- oil and gas region that 
V distinguish primary /secondary and tertiary production. Different crude types such as West Coast heavy 

and Wyoming- Mix* are distinguished by region. Each crude type is produced in proportion to its historic 

share for the regiorj. -> . 



V 



- ■ .FIGURE 4 - • 
Midwest Low SuUur PIES Coal iSupply Curve 
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COWVERSION ACTIVITIES 



• V,- 




vThe electric utility^ and ref iiriery/sec.lor^^ th^IES system ar^'embeA^ed direfi;,tly'-into the^ ^ii*al^^^^ 
equilibrium. model, la >this respect » the conversion models' are different from the supply models for coaUrc^. 
oil and^gas where only the outputs (supply curves) are directly included in the-general equrlibrrum mod'el. 

The-key to modeling electric utilities is that they cannot Inventory their product and must:produce - 
y'eiectrk^ This means that^ must own; sbrre" equipment that runs most of the^me and 

y'sbn«..equipment|tffat runs^^^^^ demand^. periods: /The demand levels for ^clxicv^y^u/ing a 

^ ^ar'-are/represented bi^?b^ load duration curve: A po'intT (x, y) on th^e Joad duration^curvej^^ 5 
shows that:for X hours during the 'year at' least >' :kilpwatts wenS^d^^ This cur.ve.:^r^mdd^l1h3 • 

purposes, 1s^ divided into three pieces: ba^ie^lntermediateVand^ kinds of generation equipment " 

that can :be used include nuclear, hydroeleotrit,. coal -fired (wjth antf^wrt^^^^^ scrubbers), residual fired, 
natural gas firedv and ^simple-cycle and. conib1ned--cycle distillate plants as welj^s newitkhnoTo^ 
: Any' of these types of 'equit)ment, other than nuclear, can operate In base, inte?fffed"late?Vr peak, whichever 
"• is nios€ economic for the util ity. Nuclear operates in base only. 

FIGURE 5 
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The major, role oT the refinery sec^-Qr is to reflect the appropriate relative prices for different crude 
types based o*n crude oil attributes/and the appropriate relative prices -for the fouryroduct classes 
based on product characteristics.- What should h/ppen is that-factors such'as the quantities of^ the four 
■ products consuiijed aiid 4Jie quantities of the different types.' of crude oils sfwuld interact in the refinery 



V A. 



sector "based on rel 
demanded.^products: shou.l 
be more expensive 



a:^ive demands and availabilitiesJ Crude oils' with greater yields of the lino re "heavily '■'^ ^ 
hou.ld Be,more valuable and the pro'ducts that dominate' demar>d, such as -o*^^ 



• A prima nfcpjtwijict^ yield from each ^ using hiitorical data . . ^Next, by parametricany • 

■ varying the yields of a Targe-scale Vefinery' ■linear program, the costs' of shiftirijg the. product slate are 
^•determined. The result, is a'model that'^-fs an, extreme .po'int rgpLre^ntcrtion of "a rif^^^ ' . ■ ■ - 



THE "INTEGRATING MECHANISM - ; v- ' - ' ^" * * — ^ . 

^ • v^. . ;;r ^ 7^ •■ • ^ ''"i: . * ' ' 

/. The solution. procedure involves inserting a step-flinotiort approxii;?tff5^^ to"'the demand function ^into a.. 

)n activities/r The^apFjroximation* 




J inear program c^ontaining 'the supply curves "and the^model s. of cor 

to the. demand function ignores the effect?s of the pr^W of ;,6ne:^ the demands for 'oth^;;pr^ducts» 

e.g., only the :naturar ga£>pVicevnffect5 natural/g^ de^^ SQlve(3%4t^r'the^ 
/; objective of maximizing the area under the difference betwer^* the d^ curves. This is>. . 

■ mathematically equivalent- to finding wheref the supply and" demand curves ""intersect (Figure 6)v How close ' 
. the prices and quantities are to being on the demand function containing the cross price effects is then 
measured- -If the linear programming Quantities are not within one percent of the demand-foinction quanti- 
ties evaluated at the prices taken from the l/near , program,- the equilibration process continues with a 
new demand demand function approximation ..-Tne^-deqiand function cfontaining the crosa. price, effects is- 
evaluated at an average between tlie prices -from" the linear program and the previous estimate of the . 
equilibrium prices; alicf ' a new approximation to the demand curve, is. constructed around this point. 



•;FIGURE 6 
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In the past, people have, tried "to - find economic equilibria by just inserting a Single /flirlirKr^point and 



successively replacing the' demand quantities with new Values from the llemand model evaluated at the deal 



-variables. The dua] 'variables seemed td oscillate, and not -converge. By inserting' ^ d^lhand- curve, 
approximation, an equilibrium is achieved after 6 to 8 iterations involving the solution of linear 
programs. Currently, we vary all prices the: same percentage simultarie9usly in calculating the step 
.widths.. If the convergence were slower, we would try varying all prices with percentages' proportional 
to a trajectory of prices from' successive iterations, '.Other procedures have been c<^nsidered but not tested. 




THE RELATION OF PIES TO GE^j^L" EQUILIBRIUM THEORY \ Y ' " - . 

The; NTOclassicaa'nwdel Of exchange may be described as follows. Each cdn^miag unit. 1^^0^-1=1 m has . . v 

IpUial-.endowment of assets: y . / . ;^ / - '. - \ . • ^ 

At: a price ♦vector ir'-;^ (1:^,. ).„conStrrain5 uait^" demands a vector of products, (it) and Tia^^an- income . . 

of'l^ = " •iJ.W". For convenience, z"^.'! i.s '\ss[any added;,4s a requirement, i-.e. , the analysi^^^^ J,-^ ^ 

. terms of -constant dollars. 'Sin^i* in equiltbriuni 1ft 5ndi*y4dual cannot spend mpre^than the revenue from the s^. _ 

iale'jo'f his assets, - : . '^^ ' ' • . " . . • ..^ • . ' : 



This leads to Walras law: 



.i=V 1=1 



That is, the monetary value of what is. demanded equals that of what -is supplied. Letting .gjCjr) =1^^ 
(d^(ul- Wp excess deiiand function 4§r*as^t j, for j=l .... ,n an economy fs in equilibrium when 

- Tliat is, either, sbpply , equals demand or supply, exceeds demand and the price of the asset is zero. When 

the pure exchange economy is general wed to include activities for the conversion of one asset into another 

using 1 inear activities; we have th^ following/definition of a competitive equil ibrium: 

* ■■ V " • ■ "^1 " - ■ / ■ 

Definition --A- price vector tt*. and' a vector 'nf. activity level s';y* constitute a . A • 
* I competitive equilibrium if: 

■,a. Supply equals demand in all markets, pr dCir*) = By* + W " / ^ \^ 

• ' " where B is the matrix of possible- activities; and ^. '^^ 

. ■ : > ■ * J'" ■• , \ ' ■ ■ ■ '. ' * 

b. production is consistent. witTii profit maximization in the ^ 

sense that ; IT. *bi-|<0 with equality: if, y*>0. . . - * . 

"Part b is a requirement that excess profits are associated only vdtli rents on scare?, resources.. ^ 

' • -- ■ , * ■ ■ ■ ■ " ■ . ■ ■ ; • - - ' ' ■. 

In PIES-'the assets are" raw materials sucrf" as coal; oil and b.foroass as well as electrical generation - - 

'equipJnent.- refineriei, pipefines, etc. ^ddet^^o these-is an aggregate asset which represen^. capital . for " 
■ ■ ■ ' - ■ >. ■ ■ •v-vvv* " -■ - y . . ■ ■ ■ •. . • .. 

new equipment and. for .developing new resouroe^,.^labor a^ other nonenergy resources. .Tlie components, of • 

• this aggregate asset are priced, in' terras. of 1975 .^ol iVs.. giving us the^objective function cost coefficieats 
"^in the linear programning subprobl em. The number^ of /aTs^^'-f^^ ts "^arge because each step on each 

regional supply ctirve of.a:fuel is. a different asset in the' exchange economy with, a different equ.il ibrium 
• price. (dual vanatle on the bound row). Every bounded vari|ble is an asset, as is every product in-every . 
deman*. region. In PIES an economic equilibrium. is found where all assets are priced relative to the' 



' Js'to an alternative 



aggregate. asset.. The prices may then be normalized to achieve Z_. ■ny^. .This leads 

, interpretation of the PIES mechanism for .searching for an equilibrium. The'goal at each itera^oiT is to.- 



^^fjL^^^^^ Wlith the dual variables' as prices and a successively Improved approximation to 'the 
, vr demand -modalC • . „ , * ■ 

• NONCOKPETITIVE PHENOhEjilA MODELED IN PIES - . < " ' ' , " • - 

There are three areas h*ere regulatory- actions that^alter the sti^ctune^'of the econdrny" are modeled. 
These are the average -cost pricing of electricity, current oil import entitTefflents and lnterstate 



, regulat^oti of natural gas; r/i: 



'i 



• Since th^ are Increasing re^^^^ transmission and distribution, the .electric utilities - 

^. constitute a natural monopoly which must be wgulatfed-V some fa$h1^ ' Currently* public utility coointssions 
.i^gula'te.utiljties to provide them with- a reasonable rate of return on their ;total Investment. {ThU means 
that customers are; charged the average, not marginal ..cost of del^yerlng electricity. The coit curws for 
delivery Idok as follows: - ' . _ ^. ' . =, . • 

.• . ■■• , . .-" - FISURE 7 : - •■■ ■; • • 




-Given a set of fuel prices, the marginal -^^^ where each step represents bringing 

into use a different kfnd of generation equlproent. The higher costs are due ta using less efftclent 
equipment or includi]«, the_capitcn costs for, acquiring new equipment. For low^quantltles of electricity ;, 
the average xost curve iis ^higher than; the rnai^inaV c This Is because the- cost of the existing 



equlpmjen^is Included in the average cost of electricity. 



The p/ices from a linear program are marginal prices. Therefore, for the demand model to respond to ' 
average instead of marginal prices an- adjustment^i^t be made to the linear program. The approadT taken is 
td approxljnate- the average co'St curve with tlje m^giriar'cost curve. This is done in the fallowing manner. 
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Say the anount of "Electricity demanded^in -a gtwen region at- the end of a linear pVogramniij^g step The 

• difference between the nwirginaV and average costs^ of electricity i-s C-D.- In Tr«jnsVng the linear B.rogratn 

" forihe'^ex^ 'iteration the transmission cost is changed so that the marginal cost'is approxitnately.-equal to. - 
the avecage cost at B. ■.Letting tx be the .adjustments to the true transmission cost ih the linear.,pro%am^ 
from the previous iteration^ the flew Irdjustjf^nt is Ic-D +'a)/2." Thraverage is. taten'.for smoothing^rpos^.-, 

.. This average -cost pricing mechanism can have 'a substantial effect on convergence.. . First, if the^ quantity 
of electricity is less than -A.' then'a decreasing average cost curve i^ approximated by a nondecreasing ^ 
Vunctjpn.; " The resuTt is a perverse behavior wljere £he, amount of electricity generated tfecreases at eaclv^ ... .. 

■ iteration while the cost increases. . We have afso experienceVanbther~ form of 
( of our . implementation. We' have seen thefquantiV of^lec.tri city o 

cost curve, caustng the correction between average ind marginal costi to not stabilize. This 

means that the marginal^b average cost,adjustn«nt i%.fTuct«at^^^^ to an oscillation in pVices awT- 

quantities. This, occurred in the Northwest, where there was a big, jump in marginal cost, ifi going from hydro 
"■poWer\o Wssil"'and/or nuclear power source of 'el ectricity . We do. not hajie a complete , 

•explanation- of '"this phenomenon. Our best guess is that it has to do with the behavidr near the minimumrof 

■ " .. ■ • . : - ■ . ;' . ■ ■ - . ■• 
the average cost curve of our implementation o? this.approach to average cost Bricmg. 

; . •. ■• . . ■. ' ■ ■ ■ ■ ,; ■ . ■ - ,,, • ■ ■ : ' •• 

OIL ENTITLEMENTS , . - . - • ^ ' '-./^ 

The. current regulations on oil production require that 'th4 average price of domestically produced oil be 

■ below $8.00.- Sinc-fe the marginal price of oil in the United States is the world, price because -of. our import 
dependence, the marginaUpY-tce of oil products^ would be' refining costs plus crude ai^costS a^t the world 
price'if . there were no ot/her pfpvisions in;^the Taw.' '.The ^regulations specify that refiners who i^se domestic- 

. oil pay $X. per barrel into a fund from which refiners who li^ imported oil 'receive' $Y per barrel . This 
entitlement to^the users of imported oil makes them ino^ifferent between domestic and imported oil s . that Js, ■ 

. where Pp is the.average domestic oil price., and Pi is the'imp^t price.. It^ is aTsp' important that the fund ^ 
never run deficits or surpl-uses, that is. . . 

'■ • . ■ xqd = yQi ■■. - ' " ' , . " ■ 

• Khei^ is the dbmestic' production and Qj is imports. There alsoVe entitlements to different, types of i 
'domestic oil . ' There is a . legal definition of what is called "old" oil and JW' oil with' ",ojd" oil having 
a substantial Ty different price from "new." oil. „ * . ' ' \ - ■ 

• Entitlements are modeled as follovte! The total; domestic oil production given the regulations is estimated. 
The -supply curves for oil in- the linear programming matrix are replaced by this supply^oint and the cost 
in tte objective function is the average price for this oil.. New activities are added to the matrix that 
essentially tax doniestic oil an amount X and give, a.credit of Y to imported oil . The X and Y are . " 



recalculated between L.P. iterations us-y^^-i the above eq^^ifions'J '^In- equiTibrium we. therefore, have a 
scheme where these equations are satisfied and a forecast of the impact of this form or regulation.. 

NATURAL GAS REGaATION . - ' * > ' . ' . ^ V 

. Currently, there are' two markets for natural gas: the unregulated Intrastate markets and the Federally . 
regulated sales of natural gas across state boundaries. By^ilaw the interstate gas price is based on ^ 
historical c6sts;^this has lead to a two tier marke$ and has developed where intrastate gas sells at-a.- :. 

■ '^v '\ . ■ • ■ r • ■ ■ . ; . • ;. . : . ■ 'v-->^ 

'higher* price than interstate" gas. Consequently, the only new sales of interstate gas are .coining from • 
onshore areas, where fields are close to interstate pipelines and there are no intrastate plp€Uite?s ' nearby V 
or from the outer-continental shelf which is under Federal jurisdiction. The supply of interstate natara? 
gas in a given year can be estimated by taking the current rate of production and reducing it by -the 
natural decline from existing wells in the onshore regions and adding. to this the production from the 
outer-continental shelf . Since the customer must becharg^^he average cost of interstate gas (there are ^ . 
three-price levels for various vintages of domestically produced interstate ^as plus the costs of 1 Iquefied 
naturaT'gas {LNQJ and imports from^Canaday, there is a shgrtfa.!! in supply in- regions where there is\l^ittle 

r 



or no intrastate gas available^ L. - . .v^. 



To dieal with the ^ortfall a priority s&heme has been developed by the FPC to allocate gas to states by 
classes for a giv6ri 4>ipeline. Each. state then allocates the gas-. avaiTable from the pipeline, Each^state 
then -allocates the gas available from the pipe! Inerto customers within the state' based on its own priority 
structure* ; , ' * * . 

- r'-'"; . r ■ . •■ - " 

I he modeling: approach, taken^s. to .assume that gas customers f-all into two. distinct gjpwps: thosewlth - 
interstate gas and'ttose. that must, use intrastate gas or a bundle of other fuels with the bundle" containing 

fuels as electricity and distiyate. To determine the , customers who receive interstate gas, the 
available domestically produced gas is allocated in an approximation to the FP(?prior'ity.sc?fiene across 
the.nation rather than by individual pipeline- The priorities, in order, are residential , commercial^ raw 
material ^nd ijidustrial . After a region receives its share of domestically produced gas, imported gas ^ 

available to the region is'then rolled in; until- either all demand is satisfiecl or thereMs no more gas 

' ' ; . • ' ^ - ' . ■ - 

available- Excess denand in curtailed sectors is satisfied fir^st by intrastate supply to the extent it is 

available at creditable pric^ and then by a displacement or switching of this demand *tp other, fuels in a ' 

way that is srensitive to other fuel prices, sector-specific end" use efficiencies and historicaT shares- 

^y for illustration/residential demand is satisfied' but only a of conmercial demand is satisfied and ^ 
there is no raw mat*i*ial or- industrial demand to be satisfied, ► 



The total supply Jto^ at! commercial customers is 



: FIGURE 8 




INTER- iriTRA- SUBSTITUTE BUNDLE AND 
STATE STATE ' INTRASTATE GAS 
GAS ■ GAS 



" The demand .for natural gas is the sum of the demand satisfied by interstate gas at the interstate gas 
-^price plus the demand for intrastate sas and the substitute fuel' bundle. Ignorijig cross elasticities the 
* demamJ for gas is Q. | a P^Wre P is the natural gas price, e is the elasticity of. .natural gas demand 
.:and arid a is a const2^ri^ Pr, the quantity of gas demand met, Q. is aaP^.. This 

rmeans that the demand curve for' intrastate ga^s.ahd.the substitute bundle is (l-a)aP|v In Figure 9 th^ 
demand curve for gas. is D'". Because the. inter- and .intrastate markets are separate, the total demand for 
na^tural gas is represented' by the demand curve D where Q = + {^^aP^. That is. the qtiantity of - 
intrastate "gas and the substitute fuel bundle demanded at, Pe is E - ,Qr. The quantity of i-nVastate gas 
■ consigned is A - Qr and the -quantity of demand met by the fuel bundle is E-A, The price Of .tffe gas, dis- 
placement ^undTe is a sector-specific, market price constructed from its fuel components. As the <^iagram 
shoi95» this activity is assumed- to dominate the further movement. up the interstate SupplyAurve. The . . 
assumption whlchjustifi^^ the process model of fixel substitution on the demand side is more 

reliable than the econometric itiodel at very high prices. ^ * 

^ . FIGURE . 9 :r V - • . • ' 
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Note_that^the_total quantity <if_gas demanded^ with_thij demand curve is greater than the quantitji^ demanded 

<" ' ' " '" * " ~ ' ' ' " ' " ' 

^ with the original demand curve D*. This 1a because consumers of interstate gas see the*»low interstate ^ 

gas pricg and not the higher intrastate price. 

^ * ^ . • > . 

St ' " ' ' • ' ' - • ' ' 

* 'Jf COMPARISONS OF PIES TO OTHHR PROCEDURES FOR FINDING A COtfgTITIVE. EQUILIBRIUM • «| ; . 

^"^^^^ '' '' 1 

The only other technique for finding a competitive equiTjtipium with computational results involves 

discrete approximations of the simplex of prices and' a search for a fixed point. The fixed point approach 

of Scarf [3] with enhancements by Hanson [3] has results that cannot be seriously compared with PIES 

because the problems ^solved are on a much smaller scale. Appendix 2 in [3] gives computational experience 

for. Scarf's algorithm. He estimates that computation time varies as m^- On^an IBM 360-50 he estimates 

the time to solve a 15 asset problem to be about 2 minutes. Extrapolating, a 100 asset problem would 

then take more than 2,000' minutes". The solution step invPIES with 100 products cpnsumed,. not counting 

intermediate products or raw materials which are in the thousands, takes from 20 to 30 minutes on a* 

370-168 under MVS with the number of linear prograrps- to .be solved ranjg,ing from 6 to 8. When natural gas 

regulation is in the model the number of iterations incfeaies to 15-20. The reason for the increase in 

iterations is that supply of intrastate natural gas is 1nelastic-\rki many regions leading to large price 

changes for "little quantity change, • * 
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NATIONAL AKD INTERREGIONAL PROGRAM>aNG MODELS OF 
LAND AND WATER USE AND THE ENVIRONMENT 

by 

Earl 0. Heady, Kenneth J ^ Nicol, and Dan Dvoskin 
The Center for Agricultural and Rural Developmert 
loWa. State University, Ames, lowa^ 



^ • * Abstract - : 

A set of large-scale pfogramming ^models , has been developed and quantitatively applied to 
analyze laiid.oise, water use, environmental restraints and impacts, agricultural ^nd food 
policies, export problems. Income redistribution patterns and othei;, facets of the nation's 
agficultur'ai sector. Perhaps these are the largest operationally useful mathematical ^ ^ 
programming models in existence. The models incorporate xid.ne land class restraints in eac^ 
of 223 producing regions." They include 51 water resource regions^ with corresponding re- 
straints and SS^.'market regions with demand relations for all relevant endogenous commodities- 
The models generate 'results at national, state, regional or looal levels. Their results- 
have been used- by ail major national commissions on food policy> water allocation, and land 
use. " . 



Many applications of mathematical pi;ogramming 
models have now been made. We have completed and^ 
underway a sec.which is. rather unique in its' size» 
scope and national policy uses . _ While we employ 
both linear and quadratic :spec if icat ion, main 
.Reliance is placed on linear models. ' These na- 
tional and interregional models applied to agri- 
culture and natural resources have been under 
evolution for a dozen yearsv: This ,time period has 
allowed us* to build up a vase data bank and add 
many dimensions to models. ; At various stages in 
their development, the models have been the main 
quantitative base for various; commissions deal- 
_ing with national policy. They have served this 
role for the President's Food -and Fiber- Commis- 
sion [3r, the National "Water Commiss ion [4], the 
-Water Resources C<^uncii^s National Water Assess- 
ment [.7]» a'nd the Midwest Gover^^ors Conference on 
Land Use [5] and for^ numerous applied^ studies for 
the Environmental Protection Agency [6], the 
National Water; Quality Commission, [9]., and the 
Soil Conservation Service "[lOT ., 

In c^ls paper, we report on one set of models 
•capable of evaluating water and land use and ' 

. tl&ir impacts on the. enyironiaent at both national 
and regional levels [S'^. While we apply one such 
model to evaluate poiteritialls In 'fenvlrcJnmental 
improvement through controlling soil loss or 
sedi;aentation from farm land, the general model 
set is capable' of. analyzing many other facets of 
resource use and environmental Impacts as these 
relate to trade, agricultural or other poU'cies. 
We also have completed models' which incorporate ^ 
legislative or^ price restraints to attain energy 
conservation [1], to promote environmental quality . 
through reduced use of nitrogen fertilizer and . 
pesticides [2]. which enhance the .environment 

•ttirough stream' flow regulations to protect wild- ■ 

. life habitats and others. 



y 



The models have the capacity to evaluate simulta- 
neously variables and outcomes in (a)national markets, 
prices, incomes and employment and (b^ production pat- 
terns, resoxirce use^and economic structure of rather 
smaller esourc'e regions, under the posed Imposition of 
alternative poi^icies or futures. We believe that 
models with these characteristics and capabilities 
are extremely Important for the future as nation- 
al, state and local entities evaluate and consider 
implementation of^^vironmental, land use, water ^ 
and other resource and technological restraints ^ 
related to problems emerging -under t;he nation's 
advanced state of economic deT>.elopme^. Other- 

' wise, the programs and iSolicies;^ Imposed by states, 
municipalities and regional planning bodies will 
encounter unexpected economic '-.effects, causing 
them to be nullified becSause they give inequitable 
distributions of the costs and benefits of the 
goals attained. For example, initial solutions 
of our models pose the certainty, that individual 
states which impose restrained paxtems of . land . 
use, water runoff^ sedimentation and technologies 
will find that, ^tlirq^ugh market impacts, , producers 
and resource owners' of other states and location 

■•will realize economic gains while those of the - 
Imposing state wdLll bear the costs in lower in- ' 
comes and reduced resource prices. Even for cer- 
tain quality controls ijaposed at the national 
level, relative returns can be positive in some 
regions and negative in other regions. 

" Models Reviewed 

■ The ^models reviewed in this" j^aper encompass the 
whole of U.S. agriculture and" the land and water 
use relating thereto. These demand-allpcfation 
models incorporate all major agricultural quality 
interaction reflecting restraints in resources ' 
for 223 agricultural producing regions (Figure 1) > 
soil characteristics in 1,891 land resource . . 
groups, water resources for 51 water supply' 
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Figure. 1. The 223 producing areas 



regions (Figure 2), in the 17 Western States, and 
demand or; cpnmodity balances'.in/30 CQnsumer /mar- 
- ket region:? (Figure 3) . The models/'wh'iHcK- in- 
corporate a transportation submodel for eonmoiji-. * 
ties abd water and product transfer activit'ies^ 
allow selection of optimal resource use patterns 
and environmental quality impacts for the nation 
in -future time periods. They also ref-lect com- 
parative advantage in the allocation of land and 
water to competing alternatives as represented in 
relative yields, general technolog'ies, env 1x007 . 
mentally restrained technologies, production^ * 
costs, transport costs and imposed environmjental 
restraints. They allow substitution of land at 
one location for water af. ano.ther locitcion a ■ » 
thousand- miles away (or vice versa). Similanly,'. 
they allow/and analyze thesS substitutions when 1 
environmental restraints are a^ppli^d to .restrain " 
the technologi^ used in any one research region. 
.Finally, they allow evaluation of various policy 
alternative^ iiit use of land and water- resources, ^ 
^nd en^ronmental quality controls interaction.' 
with, commercial agricultural •''pcj^cdCes,- export 
goals and. domestic demands iittSS^I^regional and 
national, markets.' '^s^' "'-'^ " 



_ Objectives ' ' ' — . . 

-■ ^\ ■■ ^ ' ^ . . . \. 

: Our overall objectives iu^bu^lding these"* models. " 
are to determine Ca) whether the nation fias 
^enough land and water resources to meet ^ domes t'ic 
and. export food nei^ds when ^various environmental 
quality restraints are imposed, (b) t;he optimum 
spatial^.allocation, for* the nation and ^internally 
fot each /individual producing, land andowater 
reg^.on, of these resources accordingly, (c) the, 
extent to which sacrj.f ices .mus^t be made^ in, envi- 
ronmental .quality .g^als" as other goals (food 
prices, exports, treasury costs,, farm income, . 
energ / usg , resource values, income distribu- 
t^^' '?Lc-) are attained — or vice versa, (d) the 
^ Dst to regions and ^he nation as various latid 
u"se patterns and ei^ironmental quality goals are 
attained, (e) tlie optimal Selection among alter- 
native producing technologies and land use- pat- 
terns, _ for each re^bn ^nd for the nation, *as. 
various environmental qaalitjt restraints are im- 
posed, and (f) miscellaneous impacts including 
those relating to farm size and income,' xhe 
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Figure ;3. The 30 consumer market regions^ 



discribuj:ion'of 1:he -qosCs and benefits of these 
patterns or allocations, empj-oymbnt and income 
generation" tnxural areas. 

; Environmental quality control^ ^sed on runoffs 
and- sediment transport, "f or example*^ will have i 
•direct impact -on crops ,pi:oduced and" technology 
used on et^dible lands (e.g-, slope of land and, 
'amount and intensity ot rainfall). Howev.er, these 
' environmental; re;straints..al5o will^ave .chain^ . 
• reactions'^Jin optimal land use- in regioqs lOO^or 
' 1 poo mii'es-away.Hhich are' not subject to runoff. 

aSd sediment loss, since,' - natioi/ally, a ne^S-Trurtr ^ 
- f igurarion ol comparative advantages will be ere- . 

■ ated i^elation to both the env;^ronMntal re- / 
strait^lfend'commodity and resoui^ce ifiarkets. 
Land^'w^^cbaractejistifs giving rise Co runoff 
and 'fetfdtient loss may" be required to shift to - 

-oSib^e Wages, Uvestock 4>roduction or forest prod- 
^ts— although the outcome 'also will depend on - 
.'^^Tow crop yields,^. costs aifd retum^-under mechani- 
i^cai erosion control practices. >In. the chain 
reaction" or r^glohal' interdependence relation- 
&h.ips; land.at^bne 'distant locarion not subject 
"fe erosion vhich once produce?! cotton may riow 
"^op^ally be alloca^ted to soybeans to meet^nation- 

• al ^fciifsw- while at a different a^«tion, non- ^ 
eroEib^i^land oitce allocated to ^heat , now may be 
.spe'clfied Jor feed grains as the-^national live- ^ 
stock .ration -and export demands are met. Re- 
straints on-cheiaicaL f^rtili^iers anS; pesticides 
have slm'ilaj; remote and. complex interactibns in 
Resource use among the mAy different land re- . 
gions of the nation. Generally, those regions of 
ample rainfall and irrigation water will be shif t- 
Vd towards 'a less intensive use of land while 
aSapted regions with less runoff and rela^vely 

■ less dependence on imported technological inputs 

• will tend towa^rds more intensive Tamd use (e.g., 
more grain and less forage aqd livestock produc- ■ 
tion) , ■ " 

Because of regional interaependencies, it is im- 
possible to plan -nationally efficient- uses of . 

■ I'and and environmental quality controls or- a 
,region-by-region or regionally independent basis. 

' fhes models used-incorporate interdependencies 

among the hundreds of land resource regions and 
r allow for both direct and inditect impacts among 
regions whether they are contiguous or 'distant 



from each other. Not i>nly do . they need to in- 
corporate' these intjerdependencies among lajid rer 
source regions, water tegionsand market regions, 
but also they need to allow them amon^ resources 
and commodities. They need to allow substitution 
of land at one location for water at a different 
and distant location,, since 'a .policy restraint 
which limits the use «f capital technology ^on 
land at one location can be offset by a realloca- 
tion or. extended us^ of water at another loca- 
tion—or vice versa. Great flexibility prevails 
in the national livestock ration, tffe major de^ 
mknd determinant in national land use, and. shif ts . 
in or restraints on land. use ca^ allow or cause . 
limits on grain production, in one location to b^ 
replaced by. soybean, forage or a- substitute f eed ■ - 
■ grain in another location. An efficient land 
use-environmental quality model needs to allow 
all of the^e inVer dependencies over the nation \ 
with reflection back to optimal land employment%v 
for each land resource region. • ' 

Nature of Models 

To illustrate the general nature of the models- 
involved, under the restraint of presentation- 
space and time, we use a model projecting' to the^ 

year ^000 f or- a-^opulation of 264 milljtonj free, 
market conditions with trend levels o£\ agricul- 
tural technology in each of 223 agr:icultural pro- 

'ducing regions. This model* only one in a series 

^we .are building, ^phasizes optimal land use ,pat*' 

't^rns,' agricultural water allocation, agricultur- 
al technology atwi soil^ conservation methods .under 

.environmentally restrained soil loss* (Subse- 
quent formulations or the model set includes en- 
vironmental limits on chemical fertilizers, pes- 
ticides and livestock wastes.) The objective 
function in equation (1) minimizes the cost of 
producing and transporting, the various crop and 
' livestock commodities among producing and land 
resource regions of origin, regions of process- 
ing, and regions of ;consumption. The costs allow 
the system to consider different technologies 
* (cropping or land use systems and mechanical 

.practic^> iti restraining soil loss to alterna- 
tive envirorrmental quality levels. The costs of 
water consumption and transfer also are included 
in equation (l5 - , The programming prices and 
costs co^er all ia?tor costs (except land rents 



which arc reflected £n shadow prices) ,and thus . 
allow* simulation of a'lojig-ruti market equilibrium 
"*^tyr' each.' commodity with .a. national allo^cation re- 
flecting "ch^e comparative ♦:idvantia^e of ^each o£ V 
^ ' t he "2 2 3>"pr od u c i i\g "r e g io i\'s", siiHJ e c t" ' ' t cT feri v ir o rim e n c 
' al restraints and Ch^^evel and location- of coneii- 
.mer demands/ Th'e objective function Is miTiimi-ze 
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•^L'^ere the variables, parameters, and other terms 
^ 'x^^Cv defined in a foTlOwing section. 

. . . . - ■ 



Re s t"r<i ints ■ and vari a b 1 e s 
• ~ ? — 

Each' land^ group- has alternative crop management 
systems producing commodities with associated 
yields and soil loss subject j:o the soil types, 
average, weather . prevailing and the conservation 
tillage practices>utili2ed. Data were developed 
•in conjunction with the Soil Conservation Service, 
U.S.D^A., to represent soil/ loss per acre under 
various mechanical practice^ and rotations^ or 
land us6 systems.^ .Th^ soil loss alternatives'^ are 
evaluated through "the .universal ^oil los^" equation 
(2)- s.The e<iuiit ion' used for each crop Amana^ement 
system is of rhe fbrm , . ' " 

y' Sti ^ K-L-S-R-C-P ' (2) 

where - ^ • 

SL is the per acre gross soil losis;- 
K- is^ the erodib'ility factor associated with 
^ - the so£l type; - 

L is the computed value relating, slope 
^ length tro 'coil loss .control; 
- S is the d^iyed from a^ nonline&r function 
relating>?^ope gradient to' level of soil 
loss; ' ^ 

J R is an inde3^ of ero^bility for. the rainr- 

^ fall of the area accounting for varying. 
!^ levels of Intensity, du*ration and measured 

^ rainfall; * * 

. ^C is* an adjustment factoff "g-iving an index of 
' the .relative ability of alternative crop-- 
ping "patterns IfT" reduce soil" Iqss; ^ 
> P is .an adjustmettt factor to account for the 
, potential soil loss reduction from adopt- 
ing conservation pra'ct'ices, . 

. . -■ -. > 

Each activity ^n the model represents an alterna- 
tive crop management system which' incorporates a 
given rotation, crop till^e method and a conser- 
vation ^pra^ct ice for a^i individual land resource 
-group. The^ ro'tatlon and tillage .method ^combine 
to give unique C value and the conservation prac- 

. tice determines the P, factor.. The K, Lc, and S 
factors are dependent on the soil characteristics \- 
and the regional rainfall patterns determine the 

"^R factor. ' ' - • ^ • , 

Assoc lasted with the alternative crop management 
systems 'are spec4.fic pea: crop costs and crop 



yields for erfch region. ^ The cost data reflect 
expenditures on machinery, labor, pesticides, . non- , 
nitrogen f ertilizejrs (niprogen is balanced endog- 
enous to J the modeDy and miscellaneous production 
' items . ''The^'cfompcTnchc c~6sts^^ -.7 
efficiencies ^of farming resulting frqm working 
land in straight rows, contours, strip cropping, 
witli, terraces', under minimum tillage ot under 
conventional tillage. The 'alternative costs also 
tef l^ct the higher pesticide requirements and 
lower machinery and labor requirements for-trops \ ^ 
under a reduced tillage cultivation pattern. The^ 
costs"sun 'to an aggregate which d^ends on the \ 
particular cropping management syBtism and when 
combined-^with the outputs from the -system, reflect", 
the comparative advantage of each system on each \ 
land class in each region. ^ \^ \ ^ \' 

The outputs from the system reflect x^^^^^ °f 
each crop"'^nd the associated .quantity of soil loss ^ 
per acre per year. The. iijteraction within zhe ' ' * 
system also is reflected m a nitrogen balance 
subscctor where the nitrogen flows in the model 
Jrc e^i.ntjined * The entire '.cost and yield section 
of the-, model is interlocked with alternative 
n technologies , 1-evels of resource input and alter- 
native input uses to meet domestic' and export de- 
mands. The nitrogen-fertilizer-crop yield section 
is'^ example of other interrelationships in the 
model'.' Nitrogen available in each region is an 
independent variable in the crop yield ^equation 
but the source of the nitrogen 'may vary; It can 
be supplied from chemical fertilizers, livestock 
wastes and from nitrogen fixation by legumes. The 
livestock wastes available are ' dependent on the 
type and quantity of feed available foif livestock 
and the concentration of the animals in the re- 
gion. Also affecting the yields "of the crop^ is ' 
the land class on which it is grovm and the con- 
servation and tillage practice associated with 
Che- cropping management system. 



Both dryland and irrigated crop variables are in- ' 
eluded f or ^pi^ducing tegions.in the 17 Western 
States which ''grow irrigated crops and the model 
allows selection among dryland or irrigated 
farming for each region* A range of livestock ' 
rations (variable s)^ is ay. owed in" a^l ^23^roduc- 
iug regions since the least-cost ■ feed mix can be 
-drawn from vaiffesus grain,; , forage andcpasture crops 
grote in the region or ii^ported (where allowed) 
from o theirs.- .The mbd^ includ^es variable's ;repre* 
senting. various cropping sjrstems and technologies 
aff-ecting ^soil loss, , rivestock. productioa-, commod-* 
»ity transportation, water transfers, consumer 
demand fulfillment and alternative ^export levels. S 

Each o^f the*" 223 prt>ducing regions' has land; re- 
straints of the nature'-indicated , in equations 
3t9. Each region has a soil loss rjestraint as in 
equation (10)', a nitrogei) balance equation as" in 
(11) and a pasture restraint as in (23). Each 
of the' 51 water supply regions '^as a water re-- 
straint as in eqiiation (13), -where variables 'and' 
parameters are defined subsequently*. , 

Each of the ^30 co^dming regions- has* net demand 
equations, • for all of t^he relevant crop and ^ 
liVestQck 'activities as illustrated by equation 
(lAJ. .Regional consumer <4emand quantities were 
determined exogencmsly from g€segr"aphic and ^ 



national projections of population, Wcononiic ■ 
activity, per -capita incomes and international ex- 
ports-through the region for 2000. viational de- 
mands were defined for cottoiytf?l^^ar beets as 

- indica ced-iii -equatlotr - { 15>^.4EaJ^ry-. prod 

sheep, and other livesto'cTc w^re i^gulated at the 
cdtisuming region level. International ^trade was 
regulated at the regional levels as indicated in 
equations (16) and (17). 

Commodities included in' the endogenous analysis ^ 
are soil loss, nitrogen, water, 'Corn, sorghum, 
wheat,, barley^ oats, soybeans, cotton, sug^r 
beets, tame hay, ^ild hay, improved pasture, 
unimproved and woodland pasture, cropland pasture, 
public grazing lands, forest lands grazed, all ' , 
dairy products, pork, , and beef. Also accounted 
for prior to solution of the model are other crops 
including fruits, duts, vegetables, rice, flax , 
and broilers, turkeys, egg production^, sheep *and 
other livestock. ' . 

Dryland cropland restraint each region by land 
class: ■ ^ V ' 



■ ^^^i ^i ^^i ^i " ^i^" " ^ ^^^^ ■^'^^■'^ 

'Pasture use restraint, each 'region: ^ 

-■V^--^i:^'\y^^k}i:^'i^'^-i^ 

km . n • ♦ r / • . 



(12) 



Water use yescrainc, by water, region: 
WB + WT^ + WI„ - WO - WX - WE + .Wp I 



(3) 



m 

Irrigated cropland restraint, each re 
class^ 

n . ^ 



.Dryland wild hay restrainr; each region:. 
Irrigated wild hay restraint^ each regixan: 



• IWH^ .a^ <. AIWH^ 



(6) 



Dx;yland permanent pasture' restraint, each 
region: . ' 



(7) 



Irrigated permanent pasture restra'inJ:^ each 
region: 

<S) 



forest land grazed restraint, each region: 



FLG^ a^ < AFLG^ 



, Sola loss restraint, each region, ,each land 
class, each activ^^ty: " ^ 

. ' (10) 



.^^iknrHi' - ^^iki&fn . 
Nitrogen balance restraint, by region: 
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Commodity balance restraint, each consuming 
region: ' - ' 

^ ^ ^ikm ^^ikmc ^ "^ikn ^^iknc ^^ikm 
PN,.cy,-7^L^ cy^^ ^ 0 d^) 

V\ ' • • ' . . ^ . . 

National commodity balance restraints, for cotton, 
sugar beets^and spring wheat: 

i k-m " ^ 



National export i?estraints: 
•''National impOrt 'restraints; 



ic+e ■" . c+e 
i ' ' ' * • 

' Non-negacivlty restraints: 



(15? 
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^ikm' ikn\ ikm* ip' ^ 1 ±. , x 

FLG^, FP^. ELj^, -WB^, OT^r WI^, TOJ.-WX^; "^w' ^"^V 



• -Sr. 



tc* jc* icre 



(18) 



The^subcripts and^ariables for j:he' above eqia 
^ , tions are defined in the section ^below, ^ 



Subscripts and variables. 6 f It ho model 

The subscri-pts and variables relating to the equa- 
tions in the text are those which follow: 
. subscripts ^ " • . p . ' 

'e r;-2^»;T;; ;"lS"f or ^ the endogenous ' coramodl- 
ties in the model, , * 

e* "^1, 2, . . . » 5 for the exogenous livestock 

. alternacives considered, 
g^" 1,2.3. for the commodities balanced at 
. ^ the- national leycl, 
i ^•1,2, . - . ,223 for the producing areas of 
the model, 

^ • j - l,2>-,-,30 for the "consuming i^egions of 
the model ► _ " 

k • 1,2,.* ',9 for tfie land cliisses in each . 

producing ar£ra,i ^ • « 

m « 1, 2, • . . , f or' the jdryland crop management 
systems on a land class in a producing 
area, 

n," 1,2, - . . ,for the irrigated crop management 

systems on a land class in a producing- 

area, ' , ' ' 

p " 1,2,---, for the livestock activities de- ' 

fined in the producing area, , * . 

t - 1,2, -..,458 for the transportation routes 

in the model, 
w 1, 2, . . . , 51 for the .'water supply regions 

In the model- . 

variables 

the amount of land used by the associated 
activitjr from the land base as indicated 
by the subscripts . ^ ^ 
MIPP, the number of acres of irrigated permanent 
pasture available the subscripted pro- 
ducing area; * . ^ 
ADWH, the number of acres o£ di^^land wild (non-t 
cropland) hay available in the, sulJscripted 
producing area; . 
AfLG, the^ number of <acres of forest* lan^d avail- 
able for grazing in the subscripted pro- 
ducing are5; ^ . ' 
ADPP," the number of acres of dryland permanent 

pasture available for use in the , subscript- 
,ed producing area; 
>^AIWH, the number of ac3?es of irrigated wild (hbn- 
■ cropland) ^ay available in ^the,. sutiscripted- 
. ' producing ftrea; 
ASL, the p«r acre allowable soil loss gjubscript- ' 
^d for "land .class, producing area and ■ v 



activity; 



r 



b, the units of^ nitrogen equivalent fertilia^er 
produced by livestock, subscripted" for pro 
' due ing . area and activity; ', 
cy, interaction coefficients' (yield or^us'*).pf' 
the relevant commodity ars "regulated by the 
■ associated activity and specified 'by the 
subscripts; ' ■ \ 

d,. the per unit of activity water use ccr€ffi- 
cient as rjegula^ted by the associated^activ- 
ity' and specif ied" by the subscripts; * 
DPP, level of use of ^dryland permanent pasture 

in the subscripted produci'ng 'area; 
DWH, level of use of dVyland wild (non-crop- 
land) hay in the subscrip^ted producing » 

• area; * \ 

^ lev^l of net exporip^r the associated 

^commodity, in the associated region as . ' ' 
J specified by the subscripts; " • " ' 

EC, level of exogenous crop production by 




subscripted nregj-on; * 
EL, Isevel of exogenous livestock production con-* 
si^^ent^ with ' the subscripted region; >."^ , 

EX,^ the^fi^l .of national net export for the sulr- 
scrl!ptea*^mmodity »s determined exogenous 

to"the'"irtode3 

f, the units of n^roge'n e<iuivalent fertilizer 

required by the ^associated activity and speci- 
;:■ fied by ,the subscripts; 
FLG, level of fore^ land grazed in the subscripted 

producing ; 
FP, number pounds of nitrogen eqM.ivalent fer- 
tilizer/ purchased in the subscripted producing 
area; / , 
IM, level (if national net Imports fo<i? the sub— 
scripted commodities as determined exogenous 
to the/ model;- ' * 
IPP, level bf use of the irrigated permanent pas- ' 

tur^ it^ the subscripted region; 
IWH, lev€^ of use of the irrigated wild (non-crop- ^ 
landJSii^ in the subscripted region; 
L, level of tsUc-J4^estock activity 'with the type 
and region deper^ent on the subscripts; 
LD", number of acres o^ dryland cropland available 
for use as .specified by the region and land 
class subscripts; • ^ 

LR, tiumber of acres of irrigated cropland avail- 
'/ able ^or use as specified by the region and 
land class subscripts; 
PN,' levei' of population projected to be 'in the 
subscripted region; ' 
q, units -of -pasture, in hay equivalents, consumed 
Ify the associated livestock activity and spec- 
if'ied'by; the subscripts; • ' ^ 

r, units' of aftermath or regulaj pasture, in hay 
equivalents^ produc;ed by th^ associated crop- 
ping or paisture activity and identified by 
the subscripts; . : ^ 

SL, level of soil' loss 'associated with any activi- 
^ty over the range m+n .in. the' region and land 
class designated by the subscripts; 
• T, level of transportation of -a -unit of the 

commodity either" into or' out of the consuming 
^ .regioi/ designated ;by the subscripts; 
WB, 'level of* water purchase" for use' in the water 
barance-o*f?the water supply region designated 
'hy the ^subscript; , , 
iJD,. level ^f desalting of ocean watser in the v^ater 
Supply regiot^ designated by the *^bscrlpt; 
jjWE,*- level -of water to be exported from the water 
"^-Z- stipi>l3;«> region'' subscriptedT;" 

,WT[, ie'^el of movemeri^ of w^te;:"in pr*out of t?he * 
water supply region through jthe interbasin , 
transfer network; _ " • ■ 

WOy level of water- requirement -for onsite u$es 
. ' .such as mining,* navigation and estuary main- 
<r tenarice in "the. water supply region subscrijit- f 

WX, ^level of water for the exogenous agricul- 

, T"r£il crops and livestock in the water supply 
^ ^'region subscripted; 

Xv leicel of employment of the dryland crop, man- *- 
agement system, rotatioa, in the region and 
on the land class as de$ignate<i by the. sub- 
scripts; , 

Y, level" of emplojrment of < the irrigated crop man- 
agement system, rdtatioii, in the region and ^ 
on the land c^ass as ^desiga^teS by the sub- 
scripts; ^ ^ "^^^ar I _ , 

Z, level of employment of the ^cir^Jand crop man- 
agemfent "system, rotation, on the land class 
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in the region as desij^iaced by the subscripts when 
the lartd h^s been designated i^sJ available for 
irrigated cropping patterns. 

"Illustration of Result)j 

Solution of the roOTiel provides JCndicatfon of opti- 
mai/i^^nd use In each producing, region or each of , , 
the 1,891 land resource groups at- prescribed 
levels of environmental quality restraints, con- 
sumer demand and distribution, export levels and 
other policy, or nurket and technology parameters . 
Our illustration is in t'hc case where the only 
environmental restraint is soil loss. It, also 
designates the level of production 'in each re- 
gion and the optimal flows of commodities to ^ ^. 
conri^ing regions and export- markets . For pur- 
poses of. illustration, we refer, to solucicvis where ^ 
(a) soil loss is not restricted and (b) soil loss 
; is restricted to 5 cons per acre, per year for, 
each of the 1,891 soil resource , groups and exjjorts 
are at a modest level. While land use could be 
mapped or indicated by each of the 1,391 land 
^resource groups,' we illustrate on the basis of 

the 223 producing regions only. The model indi- 
^ cates not only iilnd"' devoted 'to each crop^use in ^ 
each region and group, but also can indicate 
technologies lor e^ch such ^s dryland or irrigat- 
ed, a-lternative. rotations, conventional or reduced 
tillage meC*hods and'.'others which affect land and 
^ water. use and"s'edimentaXion. 

Figure A indicates an;(Jp'Cimal distribution of row 
crop acreage* among' the 223^ producingr regions^ 
Figure S.indioates aii optimal distrilaution'of the 
close '%Tovxi crops ^nd Figurd 6 gives the hayland 
' distribution when -no: restraints ar.e placed 9n 
soil loss .or chemicai< nitrogen ,usi&. 



for sevt?n m^'jor geographic"* regions of the U.S.^ ^.^ 
because* of time and space restraints. - While solu- 
tions of' the model were made for several s^oil . 
^ross, export and nitrogen restriction levels, we,^. 
» similarfy summarize solutions only^ f'or^two soil;fy 

loss levels, one export level and -unrestrained 
' "nitrVgen"u5e~Cexce balance within 

a producing region) . . v * . I 

^Restricting soil loss per^cre to five tons would 
■ distribute land use and technologies interregion- ^ 
• ally to reJduce-national^ soil^'loss to 727 million 
cons. Without th^ ree^triction, interregional 
land use allocations and te^nologies to meet 
export demands would generate a national' soil loss , 
3.5 times greater, or 2,677 miUl^jn tons. As Table 
■ 1 indicates the reduc.tioA in average per acre soil 
loss, as a source of sedimentation, would be. ex- 
tremely large on land classes' V-VIII which are 
most erosive. While we do not do so here. Our v- 
'models allow indication of soil loss ..changes by 
each individual region* ■ ^ 

Regional variation in reduced soil loss- per acre 
is great-. Largest reductions take place in the 
. South Atlantic -(18.2 tons per acre) and South' 
Central (11.5 tons per a'cre) regions where land 
and current land use methods give rise t^ seriotis 
erosion ''(Table 2). llie reduction in soil loss ^ 
when a 5 ton per acre limit' is imposed is atta/ned^ 
especially by a switch from coi^ventional tillage- 
straight", row farming to. contour, stripi^xr^ping 
and terraces (Table 3). There also is a signifi- 
cant^ shift "to reduced tillage, farming practices 
CO attain the .environmentally at\:ained soil, loss 
, of five tons per acre< Acres receiving reduced 
tillage*- practices increase from around 2l';million 
in* the unrestricted solution to near, 58 million 






Figure U . Location 
■>rttfich no-s 



of dryiand and irrigated row crops ^ 
oil loss restriction in 2000-. 



So^l loss 



While land use,'^ tillage methods and soil loss are 
generated in \he models by producing regions and ^ 
land resource groups, we summarize results only ' 

7 < 



acres under the five-ton solution, .Conventional 
tillage piracfiices decline from 248 million acres 
un<ier the unrestripted soil loss to 201 million- 
acres under the solution for " --ton soil 
4oss. Within the conventionr . g^P^P. . v 
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Figiire 5- Location of dryland "Srid irrigated 'Close grovn 
crops with no soil loss restriction in 2000. 
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Figure 6. Location of dryland and irrigated hay with 
soil loss restriction in 2000- 



Table 1. National soil' Io«a total and average per acre by land resource 
groups for tio levels of soil loss restriction* 2000. 



Item 



Unrestricted 
soil loss 



* 5 ton 
soil loss 





Total 'tons (idLllion ton) 


2677 


'\ 727 






Average* tons per acre 


c 








class I & XI land ^ 


'6.2 


' . 2.7 






HIE &,IVE land * , 


17.8 . 








" other 'III & IV land 


15.6 


i 

' 2.8 






V - VIII land' * 


A .28.5 


1.5 






national average 

— / \ — ^ — . 


9.9 












* 






- / 
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Table 2. Average per acre ioii. losa oy i»jor te^xon lor i^ww 
loaa restriction models, 2000. 


levels "of soil 




Region 


Unrestricted 
soil loss 


5 ton * 
soil loss 




National 


9.9. 


* «■ . o 


- - 


North Atlaii,tic 


. 9.0 ' ^ 


^ m J 


• 


South Atlantic 


21*5 


^. J 




North Central 


9.2 


9 A 
^ • O 




« f e 
South Central * 


15.1 


J . D 




Great Plains 


3.2 


1. > 




North West - 


2.3 


1. 7 




South West 


i 3.3 


2.6 




^Tabt^3 . ^Thousand ^f^^** of 
J pract;icc8 )xor two 

''^> — ^ J 


- 

cultivated land by conservation - 
levels of soil loss restriction. 


tillage 
2000. 




/ 

CoRservati.on / 
uL^xage j 


Unrestricted 
soil loss 


5 ton 
soil loss 




f- i 

Conventional tillage 


247.894 






straight rov 


233.475 


1 ion 




> contoured 


11,254 


37,lIo 




strip cropped & terraced 


3,165 






Reduce^ tillage 








strai'ght row 


21.2^9- 


24,822 




contoured 


0 


18,902 




4 

strip cropped^fit terraced 


m 

0 


13,920 





straight-fow fanrfing is nearly halved * Contour- 
ing is tripled and strip cropping- terracing prac- 
tices are increased 1,000 percent to meet soil 
.loss restrictions (Table 3). While reduced till- 
age nearly triples and very large increases occur 
in contouring, terracing and strip cropping, 
straJLght-roy methods of reduced tillage do not in- 
creJce -ioaportantly. ' 

The shift* in^Sicreages (Table 4) is partly hidden 
in the reduction of 16*5 million a^res used for 
graiS crops and a corresponding increase of only 
5.5 million acres in hay on cultivated lands 
(Table 4). Part of the production required to 
meet national demands comes from an incr^se in 
noncrOplatnd roughage ^ijpduction (permanent hay 
and pasture) . More of" the reduced acreage re- 
quired to mee^ the demand for agriciilttiral products, 
results because of the, shift in production to the 
highet cost and higher yielding erosion control 
practices*. Also a shift in acreage between land 
j$lasses puts the grain crops on the higher yield- 
ing and less erosive lands . 



Cost&^of producticCn, in conjunction with the 
, transportation^network and the soil loss restric- 
tions imposed, determine the national^ equilibrium 



prices for the commodities- Table indicates the 
relative^ equilibrium prices of the commodities 
generated by the model under the two levels of 
soil loss and a single export, level. ' Soil loss 
' restrictions have the largest effect on prices for 
commodities which', concentrate on land with high • 
soil Toss potential. Compared to absence of soil 
losa/restrictions', cotton and soybean prices in- 
c^^T^se over 20 percent while wheat and hay crops 
increase by less thjan 10 percent. The increase 
in grain -prices res<ilt in corresponding increases 
in cattle prices. In evaluating the effect of -ff 
any environmental policy alternative, the effect- 
on the desired parameter and the change in farm 
price of agricultural produdts are two easily v ^. 
observed changes in our mo.dels. • . 

Changes summarized' at the national level do not, 
of course, reflect .the effects in particular re- 
gions and. on* individual enterprises.- These, how- 
ever,^ are all available from bur ^models. The 
shift in produce tion from one region to another re- 
sults in income repercussions on the' rural commu- 
nity affected. The effect of such a shift depends 
on the degree of multiple level resojxrce use- 

r 

The data in Tables 1-5 indicate' that American 



^ REFERENCES 

1. Ovoskln» D. and jE.O. Heady. U.S^AgriculCur- 
al Production Under Limited Energy Supplies, High 

"EncrgyT»rlcesr'an^"Expanding ExportsI CARD Re- 
port 69. Center for Agricultural and Rural Dc- 

, velopment, Iowa State University, jAaes, Iowa* 
Nov. 1976, / 

2. Dvoskin, D. and E.O. Heady. U.S. Agricultur- 
al Export Capabilities Under Various Price Alter- 
natives. Regional Production Variations, and 
Fertilizer-Use Restriction. CARD Report 63. 

, Center for Agricultural and Rural Development, 
Iowa State University, Ames, Iowa, Dec, 1975. 

3. Heady, E.*0. and L.V. Mayer. ^ Food Needs and-^^^ 
U.S^ Agriculture in 1980. National Advisory ^ 
Conmiission on "Food and Fiber, Washington, D-C. 
Aug. 1967, 



Model of Sediraent and Water Quality: Various 
pacts on American Agriculture.^^ >CARD Report 6 
Center for Agricultural 'and Ri^ral Development 
Iowa State University, „ Amcs/^ Iowa , . July ^1976 , 



4. Heady,. E.O. , H.C. Madsen, K.J. Nicol, and 

5. H, Hargrjove. Agricultural Water Policies 
ax«d the Environment. CARD^Rep>ort 'A(5t, Center 
for Agricultural and Rural Development, Iowa 
State University, Ames, Iowa, June 1972: 



5. Huemoeller» W.A., K.J^. Nicol, E.O. Heady, 
and B. Spaulding. Land Use_:_- Ongoins^ Develop^ 
ments ii3L_the North Central Region. Center for 
Agricultural and 'Rural Development^ Ames, Iowa', 
Nov. 1976. 

6. Madsen, H.C, K.J. Nicol, and ErO. Heady. 
Environmental Impacts and .Costs in' Agriculture • 
in Relation to Soil Loss Restrictions and Nitro- 
gen Fertilizer Limitations. Report to the U.S- 
Environmental Protection Agency. Center for 
Agricultural and Rural Development, Iowa State 
llniversity, Ames, Iowa, Nov. 1973. 



7, Meister, A.D.; E.O. Heady, K. J.- Nicol, and- 
R.W. Sfrohbdhn. U.S. Agricultural Production in 
Relationn^o AlterriiClve Water, Environmental, 
and Export Policies. CARD Report 65. Center 
for Agricultural and Rural Development, Ames,v^ 
Iowa, June 1976. . " 



8. Nicol, K.J., and E.O. Heady. A Model for ■ 
Regional Agricultural Analysis of Land and Water 
Use, Agricultural Structure, and the Envirpnment: 
A Documentation. Center for^ Agricultural ai\d ^ 
Rural Development, Ames, Iowa, July 1974. ' 

9. Nicol, K.J. and E.O. Heady, v^^e Economic * 
Impacts of Abatement Control of^ Irrigation Re- 
turn Flows and Pollutant Run-off from Nonirriga ti- 
ed Agriculture. PB 248 807. National Technical 
Information Service. Springfield ,v Virginia, 
July 1975. . ^ 



10. Nicol, K.J., E.O. Heady, and H.C. Madsen. 
Models of Soil Loss, Land and Water Use Spatial 
Agricultural Structure, and the Environment. 
CARD Report 49T. Center for Agricultural and 
Rural Development ♦ Ames, Iowa, July 1974.. 

11. Wade, J^C. and E.O. Heady. - A National 




> 



Ta'ble 4, National^ prolluccion of rgv crop* .'^oloac grown crops ^nd rotation ' 
roughage crops for two levels of soli loss restriction, 2000T^ 
• ' ■ ^ . \ ^ ■ 

y * • Unrestricted 5 ton 
Land-'use ' .■ — " — 



soil loss soil loss 



Acres cultivated (OOO) 269,113 ' ; 258.882 

Row crops (000) ^ 148,226 ^1 136.035 

Close grown crops (000) ' 75,535 \ ^73,478 

Rotation roughage crops "(oOO) 45,352'^' 49,369 * 

Non-rotationr roughage crops (000) 303.060 - \ 3^0,697 > 

s • 

• : * * 

/r. 1 . 

Demand levels are based on projected per ^capita food consumption level 
284 Tnillion people in 2000 and lnt%rnati6nal trade of grains equal to^the 
1969-1971 annual averages. ' ^ • ' ^ » - 

. '. - * " '> ,f , . 

Table 5- Relative farm level prices f or' sotnc' agricultural commodities with, 
two levels of soil loss restriction. 2000. V 

,^ i ' Unrestricted * 5 ton 

Co5»odi<:y- _ . ^^^^ ^^^^ ^ soil loss • 

/ ^ ■ " • ' 

. "Corn ^ . 100 i# - . ^' ' 107 

Wheat' * 100 103 

Soybeans 100 , - "115 - 

Cotton . ^ 100 - . 112 

Ha^ • - i * 100 ' 101. 

Cattle , ' . * 100 \ 104; ' 

Hog^ 100 \ . . 105- / 

Milk- ' 100 . ' ■ . ' 1.00 



^ ^ ag^^iculture has greax capacity and flexibility in 
/ adapting' to certain environmental quality goals. 
Bly Shifting land u$e aioong the many produclng*re- 
gions aod landv resource groups* in terms of their 
comparative advantage in yields, coimnodity "costs, 
location, and transportation, national and <:egion- 
al demands can be met "without large increases in . 
, food prices and costs for consumers at the eaysort 
level examined. The le^el of exports- per se may 
have greater impact on consumer food costs than 
does a relatively wide adaptation^of agriculture 
and land use to environmental quality goals* We 
will, however,' provide- quantitative analysis of 
these possibilities, aloitg with other environ- 
mental quality practices, in upcoming presenta- 
tions!^ ' / " ^ . , 
y * 

• ■ 
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I INTRODUCTION " ♦ ' . " , 

•" . - ^ f - • * 

Humaa dl^ problem^ fall into two major .cat e-^ 
-.gor±e^ known as food plafahing and meal, planning 
^ p^robl^ms. Both' problem area^ have been •shows to 
be amenable »to mathematical definition,- formulation 

iand solution. • ' ' - 

• ■ - . • ^ 

?ood planning is coiicerned_w±th decisions as- 
to which food entity, 'and how much^ to purch^e 

ject to given budgetary, nutritional, ^nd accep- 
tability „ requirements. The firsjt statement of . this 
in. the context of the cost of subsistence 
to Stigler [17] • This was later reformulat- 
ed b^v^ahtzig [lOX^asthe classical example of a 
linear programming model , and was refined later 
x^ith* respect to* consumer acceptance by Smith [16] . , 
The concept is -still in use in terms of food groups 
in connection with.USDA family food plans [13]. 

•MeaJ^ planning is a decision problem of find- 
ing an optimum sequence of meals consisting of com- 
binations of prepared foods, called menu items, to 
be eaten- by a person or a population, -such . that -the 
reqxxired structure of the im&als-and given- budgetary, 
nutritional- and food product^ipn specifications ar« 

' met . The entities of 'meal planning are menu items 
of '^own portion sixe, with the food ingredients' 
per portion defined by the*' respective recipes. 
This problem was' first ide^aified and solved efs'a 

, mathematical pT^ogramn^ng proTslcm by Balintfy [2]. 
The first approach and some -of the later refinements 
[3] were considering onfe meal '(or day), at a time; 
using what iS'. now called a multistage menu schedul- 
ing" ^algorithm. Each meal was a ieast^cost (best 

- buy) • combination of itens selected on the basis of 
avoiding incompatibility of .items between meals and 
within meal s^. The former rule was effected- by 
requiring a jjiinimum separation of meals between . ^ 
consecutive appearance of the same item 'or the 
same Tg'tTiri of it 6335 on* the* schedule. The -iat£er 
rule was observed by restricting., items from the 
'same attribute class to occur in the sblytions. 

The concept of minimum separation of^ items 
was not only useful in" menu scheduling as a safe- 
guard for variety, i.e., acceptability, but it-* 
could also he used to establish upper bounds' on 

" the frequency of items in a giveil time perio'd. 
This realization led to. the develo^Jment of a ^ v 

. bounded linea'r^ programming model to meal planning • 
[5] which defined in a Single stage the optimfa 
(least cost) frequencies of menu items" for" a period. 



called a menu -plan, whi^ later could be. scheduled 
into a sequence of mea^P. ' ^ , . 

Initially, -both versions of^ th^e modeling ^' 
approaches to meal plajming had cost-minimizing 
objective functions and assured acceptability oper- 
ationally only by variety and entry restrictions* ^ - 
' The cost, minimization objective va.s selected pri- 
' mardlly to show the economic 'impact of mathematical 
optimization as opposed to conventiojial methods. 
Another rationale for cost minimizing was the 
paucity pf data and methodology to represent food 
preferences quantitatively., as meal planning object- 
ives. Indeed, Cost savings from 10-30 ' ^)ercent o*f • 
food cost -have been achieved in a variety of appli- 
cations 19,11].^ 

The^e applications were initiated in hospi- 
tals, although ^ variety of other institutional 
feeding programs, such 'as school lunch service, 
college food service, as well ias nursing home,^^ 
a detention home', and military food service^' opera- - 
tions also^could'vitilize a better app.roach than 
the prevailing; conventional method of menu plan- 
ning. Such methods cannot take, into consideration 
explicitly and'^iuanti^tively the populati'On ^ref-* 
erences,^the nutrient, composition and; the coSt of 
menu iteins,- and .hence the resulting conventional * 
"Cienus are ipso facto subqptimaX and often infeas- 
ible relative to the stated objectives and con- 
straints^ [f]l 

The role of food preferences .has been long 
recognised,' and food preference and preferred- 
serving-frequency data have been routinely collect- 

^ ed in the food service industry. Such information 
was, however, utilized only subjectively in menu 
planning, mostly because the functional relation . 
between t&e preference, for an ^tem and its serv- 

^ing frequency was not recognized. Benson [8] was 
the first who represented this rela^tibn by fitting 
data to a polynomial function, but no attempt was 
made to use his results, either in menu planning or 
in mathematical .programming formulations. / 

Major developments in the last years have- 
taken place in the mathematical modeling of food 
preferences by the discovery of the .^existence of 
time -related preference ftmctions. The impact of 
this new development on- the modeling of meal plan- 
ning decisions as mathematical optimization prob- 
lems- is investigated in the sections that follow. 
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II MATHEMAXICAL FOBMULATlON-t)? FOOD PREFERENCE 

.PUNCTIONS^ ' . « . 

•It-TLs assumed - as, is observable^ in reality - 
that no'st- food *is consumed in discrete portions at 
discrete points in 'time. Consequently , 'one can 
rel^ate a. measure o£ satisfaction with the event 
that a fixed ^quantity of- food* is consumed at a. 
.given time. One can. alsa assucie that jfor foods 
that'-are fantyiiar and are d5re 05" less routinely 
consuacdT'by ap. individual-, satisfaction with foods . 
is not, only ftsrpetsienced bu^alsp- anticipated to such 
a degree ,that a measure of" utility can be elifcited 
by collecting preference ratings for a set of foods 
oa some centered scale. ^"In this investigation, 
t^ei^efore, it will be assumed that preference rat- 
ings are estimates of the measure of- satisfaction 
of an individual with a food- The word food is . 
used-here as a collective term applicable to the - 
special case^ *of"both food gtoups and menu. items 
as the case may be, . ■ \ 

Let h(t) be the preference rating of an indi- 
vidual for a given food item at time t where t 
is jaeasured from the last t-ime whto the item was« 
consumed. Clearly h(t) is ^ function for which the 
following properties can be postulated: 

(a) h(0) - since zero time interval 
"between eating a fixed portion of a food 

is infpossible as well as intolerable, * 

(b) h(<») a; where ^'a'V is a positive con- 
stant expressing the preference for a- 
known food itefii when it has not been 
available (hence not consumed) for a 

~^ long time. It is possible that "a" is 
' ' , itself some function of absolute ti;ne 

reflecting shifts of taste or seasonal-?- 
ity,, but these effects are not considered 
here. , ■ 

■ (c) ^h[at, + (i-a)t2] iah(tp + (l-aDhCt^) 
_ J'r for 6 £ a £ 1 and 0 < t^ < t2 < 

which means that tlie preference-time 
, / function is assumed to be concave. The 
evidence 'for this assujnption is indirect^ 
but- convincing. The concavity of h(t) is 
.consistent with the observation that most 
' . people ^t end to. separate their preferred' 

food items on the timer scale with fai^Ly • 
equal t±&e intervals as purposed to clus- . 
^ tering them on a succ'^s'iqn of meals. 

' ' d ♦» " ■ i " - 

-(d)' ■jj.[h(t)/t] - 0 at some unique value of 

t -■TQ,: -(0-<rTo~S'"^T^^^.e^^^^/F"s(t) . 

is the_ preference func'tioji averaged over * 
time, and it is postulated that this 
timer^average has a unique maximum at _ /• * 
time Tq._ This propertjr of g(t) is sup- 
* ported bv the evidence that people can 

estimate values of T^- by the ability of 
responding to .questions .such -as '*how f re- 
• . quently -do .you like to eat this given 
item?"' • ' - <■•■ 

- •"■ > • - 'Sl 

Empirical verification of the above assump- 
tions is available in a report: Modeling Food 
Preferences Over Time'^^ES] . It was -'found that gre- 
ference for a. part^icular . menu item can be. best 
described by the " recursive formula ^ 



(1) h(t^)-f(tn-tij.i) - e 



.r(tn-tn-i) 



[f(«)-h(tn-.i)] 

* » ^ , ■* ■ ^ 

where r > 0 and indicates tfie absolute time 

.s^scdie when the item was consumed n times before;- ■ 
Figtyre 1 shows the analytical form of this recuf- 

■ siveivr elation in terms of parameters estimated from^ 
observed data. *' ' -^^ 



. Here;^ 

^ (2) (Vtjj.i) » f(t') - a- 



be 



-ct • 



where a > 0,^ b >. 0, and c > 0 are parameters of a 
first order differential equation which -postula-fes 
that the rate of increase' of preference in" timer is 
proportional to the effect of "monotony" expressed 
by the [a-f (t ')]• difference. ^, ' 

Substituting (2) sfnto ' (1)"^ and letting?* 



Lim. ■ h(tjj) 



•-h(t) 



-(1) reduces to 
. . (3)\ h(t) 



> be 



-ct • 



L-e 



This is the expression of preference-time rela- 
tions' with the assumpticJn that the item is repeat- 
edly consumed at id'entical . t time intervals. 
, From (3) the time averaged function of pr^'^er^nce 
is obtained as ' . . * 



(4) . g(t) - |- 



be 



■ct 



t(l-e ' ) 



Figure 2 shows the shapes of ^the f(t)» h(t) ^ 
and g(t) functions 'for a particular item "as rated, 
by one subject. It is seen that f(t)- > h(t) for 
all values of t and g(t) ha:s a unique maximum at 
T_ » 5 days. 



The authors''^"^arl:^er report [6] has shown the 
methofc, and the- results of estimating the parame-^^^^ 
' ters' of the h(t) function from questionnaires.^-^^^''/^ 
It is noted, 'h'owever,* that one of the four paiSmfe-i 
ters, r i^*is needed basically*" only for the compu- \ 
tation of the recursive relations in (1). For con- 
siderations where the time intervals can be regard- " 
ed as equidistant,^ a direct analytical expression 
for H(^) can- be attempted with more economy in 
^arimeters .* For this reason an approximation of 
'h'(t) by H(t)^ is introduced in the form 



(5) H(t) 



ut 



It is obvious that H(5J—iatisfies\ conditions (a) 
through (d) stipulated ^fopr preference-time func- 
tions. Moreover ^/it is i>ossible to estimate the 
parameter of H(t) from that of^^the h(tX fimction. 
Conditions (a) and (b) are clearly satisfied at 
the same' value -of "a"Vfor both functions. By im- 
posing two additional conditions, the two other 
parameters of the H(t) function can be- uniquely 
define^. These two conditions are as follows: 

1. Requiring that both functions , have identi- 
cal zero crossing, i.e. - 



h(t) - Hv(t) - 0 at. t 
«Thls Is satisfied if 

,(6) a--^«"^'° - ^ 



-rt- 



ut^ 




The value 6^ t^^ can be determined by solv- • 
ing the implicit nonlinear- equation 

^(to> - a(l-.e""o) . tje^cto . 
by some numerical analytical method. 

Kfiquiring that. the time averaged -preferences 
have maximum at the same t • Tq time for 

both functions*' As in expression (4), G(t) 

is defined as . . ^ 



(7) 



G(t) - f 



ut 



vfl 



from <iata. ' It is' of some practical si^lficance 
that; conventional food preference questionnaires 

* are sufficient to estimate these tvo parameters. 
In most of these" questionnaires the subjects are 
aSked to indicsfte how frequenj:ly they want i 

given item^ and they also have tq^ rate the'i? 

erences 'for the item on some hed^nic scale.* With 
the assumption that their preference rating is con-^ 
ditioned by the estimated time^" 'interval Tq, corre- 
sponding to, their frequency rating, the preference 
ratings can be regarded as direct estimates-of ^ 
pC^o)** i.e., a point on .the preference- time fimc- ^ * 
tion. This assimrption, of ^ course, can be made oper- 
ational by tjhe appropriate phrasing of the ques— 

. tions. 

\ With the estimated' values of p(Tq) and Tq avail- ■ 
able,^ XIO) is 'one of* the conditions, and * 

dt t • -'^ ut 



Then the condition that 

0 /at t - T^ 



f,G(t) . |^g<t) 



dt« 



implies that 



V4-1 



4 

i.e. 

(8) 



uT 



.v+2 



v4>l 



. where T^ is already determined In the esti- 
^ mation of h(t) and g(t). 

By meeting these two conditions,, the values of 
parameters -v and u of the HCt)' function are ,^ 
determine, since expressions ^6) and (8) yield^ the 
implicit ftrm . ' 

^o V . 




which has alunique solution for v and' by substi-' 
tution into/ (6) and (8) the value of u obtains. , 

3 shows the tabulation of the estimated 
of the hCt) and H(t) functions as ob- 
frorn^ the ratings of two. selected subjects'- 
for an assortment of dessert items. The last three 
ocolnmns are obtained from the first four columns, 
.with -tfhe, exception of Tq, which is the subjects' 
preferred time interval for the/items and is 'part - 
of the input data. , It is noticeable that for most 
items the value of v ^is fairly dose "to one. 

. ■ , ^ / ■ ■■ • ■ , _ ,, ■ 

For reasons oJ^^analytical simplicity, the HCt) 
function wSlI be ii^ed as the analytical model of . 
preference-ltime rl&Lations in ^the following parts of 
this paper. ^^One?^can, however, further simplify ^.the 
function by aS^uming that v 5 1 'for most items..^ 
This way a two-parameter approximation of the p.ref- • 
erence-time- function obtains in the form 



(10) 

~ where parameters a 



p(t> - a - i 



and u 



are to be estimated. 



(11) 



is the other condition; that the parameters^ a and 
u must satisfy. Botl>' conditions are^ satisfied if 

u -. 2/aTQ 



(12) a - 2pCTo) 



The analytical properties o*f this ;s£mj^f ied . • * 
preference-time function imply .that the estimateid ; 
preference at t » » is twice as much as the^, prefer- 
ence at Tq time interval. It is interesting that " 
the observed preference-time data (Figure 3) are - 
not too far from satisfying this property,* since . 
the" estimated value of the parameter v was . . 
fairly close - to one. ^ . . 

"In the previous part the preference-time func- 
tion was introduced as a measure of satisfaction^ if 
a fixed portion of a food is consumed by an indi*^ 
vidual at identical time interval^ of length t . 
Here we cons icier a particCilar food item (or menu • 

-item)Kdenoted by. the subscript j»^. and will util- • 
ize tlie H(t) and, G(t) functions to get an efxpression 

• for the measure of satisfaction over the whole plan- 
ning -horizon of a menu plan.' 

Let N be the total number of d ays included in* 
the menu plan. With a serving time in^enrval of t 
days, the number of times menu item' j- ' is offered 
is denoted by Xj . With the standardized portion 
size for which the pref erence-time functions are 
evaluated, the following idenrity holds 

(13)^ ^ N/t • ' ^ - ^ ' - ^ 

Consequently, the preference at 'each consumption as 
a fiinction of the frequency, the ntjmber of times ' 
item j £s offered on the menu plan^ is 



^(14) H(x^) - a^ - (l/b^)(Xj/N)^j 

The total preference derived by offering menu item 
j Xj" times is obtained by miStiplying R(Xj') by Xj 

to yield G(xj) . . \ . ■ 

(15) ^G(xj) - ajXj - (i/bj)(xj/N)''j -Xj 

or more simply ■ , . ■' 

(16) . G(xj) - a^Xj - hU • xp"^^ 



ERIC 



99 



111 



bj (l/bj) 



«d by one individual. •<?Cxjy 



*w It is readito seen ithar,JS(i^l is a unimodal 
function of xj , iSplying' 't^t^ jLt_has_aJjJi3fique .jnaxiT^ 
snii^ aC sooe value of Xj, This is^lies chat indi- 
viduals ^jtend to pace tneir con^usnptlon. of food j 

' on the time scale suoih that -the preferred- number of 
.tiiaes Jjk the cycle length N is given by the- maxi- 

. mum odW^ G(Xj)^ function. In oth^r vords^ for 

^ *menu j ^ , G(x^)^is th6 preference objjective 

f uactioa»-<o be ^max: 

" i6 ^quadratic* function if Vj * 1. 

Until nov^ G(xj)*was tacitly assumed to be the 
preference .function of a given individual formfeed 
J • * it is very likely that different individuals* 
may have nouiden^ical preferences for the same set 
o^ foods. Consequently, * the notation G^Cx^) will 
be introduced ,^s the preference ftjnction of the 
i-th individual of a given population "for food i-tem 
3v . • . 



In particular 
(17) G^Cxj) 



^ij^j - ^ij^j 



^ij 



+ 1 



wiH replace the notation used in (16) . The param- 
eters a^j > and vij can be established from 
questionnaires by the earlier described methods. 
Cleajly , the G^ (Xj ) functions are concave for each ^ 
tnSiividual i , so one can express th*e preference 
function of a set of individuals H , for item j 
as follows: • ^ - 



(18) 



I Gi(xj) 



where Gy(x.) ^will have a unique max-fTmrm at some 
value ot x^> since th& sum of concave functions is 
also a concave function. Thus, one can say that/ 
for a given population the, preference realized from 
food item j. 'is a maximum^ for Xj is equal to Xj , 
where 



(19) 



G^(xj) - max 



X. ^ ''i^^^ 
^ ieM 



^ 9 



f *'This is not 'to Say, however, that the maximum 
of Gj^(xj) functions of individuals in the popi^a- 
tion, especially if the population is .very hetero- 
geneous, with respect to their preferences for 
foods. This problem can be resolved by partition- 
4King the set of individuals into subsets, '^dusters ^ 
^ such that the within cluster homogeneity of indi- " 
\ viduals is ynii^mrm with respect to the set of 

food& under consideration. The smaller the clus- - 
ters are, the more clusters are needed, and the . 
cluster Tnayfimim ^m^^I^ will be .closer and closer 
to the ^T^fTmrmg of the G^ (xj y functions within the 
clusters. It suffices to say here-that for any 
number of desired partitlonings of set . , tech- 
niques^of duster analysis are available to find 
the ni&s ^homogeneous set membership of clusters and 
^ thus, the -correspond ing values of the G^(xj) .£unc-x 
tions for any set ^f ^ood interns. . 

The/function Gjj(acj), as d'efined by. (18), needs 
3xM parameters for its evaluation. Parameter 
reduction, can be performed^ dh G^Cxj ) , too. 



EMC 



Z - I bijxJJ] 



By noting that 

v. 



^ X.J - I (v. In xO^'/n! 

'the expression for G24(xj) can be written- as 
. ; " T^tZ^ij =■ I^>ijI(Vj In Xj)VnI 

i i i . 

By denoting*?. w^-j« ICb^jV^/nl) 
the above expression reduces to 




J 



(20) 



We note that, the summation in the above expression 
forms -a mono tonically decreasing convergent series!; 
and ^ence an arbitrarily accurate approximation to 
the Gjj(x.) function can be obtained by retaining a 
finite number of terms in tlie summation. Although 
no' tests have been performed to observe how fast 
the series cohvergeS, it seems that for -large M , 
much fewer than 3xM parameters need' to be used to 
obtain an accurate approximation to the function. 
It is realized that computing w^^ initially uses all 

■■ Che original parameters. However, this need be 
done just- once, and it can be done before the param- 

^eters are actually put to use 'in the* ndixlinear pro- 
gramming model described in the next section.^ 

Expression (16) for G(Xj) was obtaine.d from 
(5)*, -which was obtained from (3) to. effect an econ- 
omy in the number of parameters in. the function. 
. One can, of course, obtain th^G(x-i) function di- 
rectly from (3) , .usliig all of. the four parameters 
in it. The resulting function is * . 

(21) GCxp- axj -.^fpF75?j . 

where c^-Nc and' r'"Nr. Figure 4 displays the form 
of the G(xj.) functions as obtained from, a Skylab 
astronaut for two entrees. 

Ill ITONLINEAR PROGRAMMING SOLUTION TO Tffi MENtJ 

PLANNING "^PROBLEM^ v • ' / 

The fundamental problem of menu planning is to 
^ define which' menu items and how many times should ' 
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. appear in a given time period-"- called a' cycle - on 
.*.lth« menu; According to the definition 'ad^op ted here, 
menu pl;mn1.ng is' a decision problem, which ^concen- - 
trates on" the whole**cycle, and atteapti^n:o find an 
optimum plan In terms of finding optimum frequen-. " 
cies- of the items under consideration. In contrast, 
menu ^cheduling'is "def ined~^is~V*problem' of deciding 
which JLtem should Appear in which meal 2nd day. . 

• * 

"^•Scheduling and planning, of course, are intim- 
ately related in the sense that the time aggregate' 

. of the menu schedule for a cycle is the menu plan. 
In tutti, a properly structured menu plan can be 
par'titionedy i.e., scheduled into a sequence of 
menus. The presentation of the material will be 
based on this latter principle. For the sake of 
conceptual clarity, nonselective menus will be con- 

r sidered *-first. 

It is assiimed that a set of n' menu items is 
subject to menu planning decision such that for 
each item j , optimum quantities Xi,^S3y:^l , 2 , • • • ,n) 
should be detei;mined for a given set of J^ndividuals 
and for a menu cycle of-* s days. In this context 
the meaning of Xj is the nximber of unit portions of 

. menu item j - to be allocated on the menu during s , 
days- It is'assumi^d further that the set of n 
menu items can be {partitioned into K subsets 

; according to the . course structure, such- as entrees, 
starchy vegetable, etc. of the meals for a day. 
Let Wj^ Gt-1,2,---, K) be' the relative weight of^ 
course k in the total preference of the meals. 
With these notations , ^the total preference of a s^t 
of individuals M for n menu items can be ex^ 
pressed as follows: - > 



(22) GOO 



k-1 i-n. 



is the n-vector of Xj com:— 



where n^^O, nj^-n, (nj^- n^^j^) is the number of menu 
items in course k , x X ' 
pbnents, and V, 

i-2 i 



(23) Gjj'(xj) - I aijxj- 
icM 



as in (18), or. as defined in (20) to achieye"^ param^ 
eter economy- Consequently, G(X) is a weighted 
additive function of nonlinear expressions which' 
all depend on the aggregate preference-quantity' 
functions of a population f or "each of the . n items 
involved. Inasmuch as the objective of menu plan- 
ning is to select menuj. items for a cycle of s 
days' which will be most preferred, this objective 
can be reached by.finriing the maximum of the. G(X) 
function. To insure-, however, that the resulting 
jvector X is ' appropriate for th^ purposes of sched- 
uling, as well as f-Am the point of view of other, 
such as budgetary, nutritional and compatibility 
considerations, only'the constrained maximization 
. of^G(X) will provide;, in general, acceptable re- 
sults . This i^ads to a nonlinear programming 
formulation of . menU.'^ planning./ 

• Accordingly^ menu planning with preference 
maximization objective can be formulated as a non- 
linear program problem stated as follows: . 

(24) max. G(X) s.t. c^X £ c^ 
AX >_ B, MX S, RX _< D 



where 'i 

G(XJ0^is the nonlinear objective function iden- 
tical to (20). '4. • 

X ' ' ■ ^ . • ■ 

c is the n-vector of unit portion costs- of 

-menu items. ' 

A . is, the mxn matrix of the nutrient compo* 
sition of menu items, with a^lx element 
indicating the amount of nutrient i ' in 
one portion of menu item^ j . • 

B is the m-vector tf^'the nutrient allow- 
* ances for some reference person for s 
days. 

M is a Kxn incidence matrix cdntaindLng 

staggered rows of unit coefficients corre- 
' sponding to the availability of the iteo^ 
for given courses. 



is 



2s 



a K-vector of components s 
for nonselective, menus indicating the 
^ number of items needed for a course, for 



f cycle of 



s days. 



\is an Lxn matrix of coefficients for 
assorted attribute cons traj. nts , propor- 
' ^ " tYonality constraints, production con- 

straints, e"fcc. which define feasibility 
conditions for scheduling the vector X . 

D is an L-vector defined by the constraints 
above. 

f 

X is ^ the vector notation for the menu plan 
which is fully defined by the values of 
the components of X . If the j-th com-*;, 
ponent of X in the solution is not zero, * 
Xj represents the number pf pcSrtions of 
menu item j . to be allocated/for s days** 

The above definit of Xj and its role with, 
respect to the feasibility of schedulii^g requires 
that all t* the components of X be integers. 
Sttictly speaking, menu planning is a nonlinear and 
integier progcginming problem. Such problems are ^ 
still consiiEfSred intra cCable^n theory .r In prac- • 

* tice, howervker , the problem is -liot . too serious be- 
cause of two favorable factors. Tirst^ s can be 

- rather large. Sixty day or 90 day menu cycles are 
cotmon, so the number of portions to be represented 
by the Xj components can be large* integers where 
the effects of rounding are relatively minor. 
Second, the nonlinear programming problem posed in 
(24) is well suited for solution by. piecewise lin- 
earization and convex separable programming tech- 
niques where the ^^id points of the linearized var- 
iables can be conveniently .selected to coincide 
with unit portions. This way all the upper bounds 
of the auxiliary variables will correspond to inte- 
. ger values and experience with such upper bounded 
linear programming models for menu planning has 

fk. shown that in such cases '.a sizeable majority of the 

™ boupds tend to bind, and thus most of the compon- 
ents of the' X-vector . will b^T^i^eger valued. 

It should be mentioned- here that an alternate . 
nonlinear programming formulation of menu planning' 
cSn- be derived from (24> and considered as useful ' / • 
for institutional feeding programs where the man- / . 
agement obj ective is to maintain a given food 
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preference level at minimum cost. This verk'ion of 
the problem is explicltely staged here for further 
-reference: , I ' / ^ "1' 



^25) Min.^-cX 
AX B, MX 



s.t 



G(X) uo' 



ferance 



^ Here Uq is some minimal level of pre 
he. maintiiined while the rest of the notatipn^ 
the same as in e^cpression (24). VThis 
similar, but not* superior , to the '*best -buy" 
programming model^s;:* discussed in [5]> where 
linear constraint :is replaced by a. set of upper 
bounds'. 



struct [ire 



ttxe 



The nonlinear programming problem (24) , after 
the addition of slack jand surplus variables can 'be 
written in the form i 



(26) 



max. 



C(X) 



K - nv 

k-i 



I: 



to 
meaji 
is 
linear 
non- 



1+1' 



GM(x-j) 



•.s.t/ I qi3>^- df i-1,2,--, nrHCfL 



' The above problfi^, although it is nonlinear 
'and large in size,.; (well over 50^ constrainds and 
400 variables for food 'service establishmeiajts) , is 
amenable tof. efficient solution techniques £n exist- 
ence [13]. The objective function/ of problem (26) 
is additively separable, and thi^makes the appli- 
cation of what Wolfe [19] has termed grid-lineari- 
zation particularly efficient. / 



features of a 
r the solution of 



We recapitulate briefly 
grid-linearization algorithm 

the nonlinear additively Separable problem (26) 



An initial set of grid points {xj^}^ 



} is 



defined for each variable 
ized program in^ the variabli 



Xj , .yielding the linear- 
il^s 0^.^: ■ 



(27) 



^Y.\ .n^ 
.1 ^ t 



k-1 J-nj^„3^+l t 



I ^jt Gwt^t) ,f ^ 



s.t- 



t. 



jt ivP^ 



/ : ^ 

for all "3 
for. ally j, t 



m+K+L. 



If the initial number of grid^ points for each 
variable , in ptobiem (27) is large ^ an acceptable 
approximation of, the nonlinear pbj ect ive function 
may. result. However, if fewer -grid /points are used, 
new. grid points can be created in the framework of 
the solution algorithms / 

k ^ - ' ' f ■ • . ■ 

If, at iteration r , the linearized program 
is solved, an optimal solution {X^^} and simplex 
multipliers (H , H^)' are avalLabli. . For any vari- 
able" Xi , a new grin point is- soughJC such- that it 
produces the most negative .reduced cost factor ». 
This corresponds to solving the unconstrained 
problem 



■- :(2k) Bin., l'^^. 

Note that. if" Gj^(Xj) 'i,s concave, cfie above ftinction 
. in xj is convex and the minimum is unique. The 
minimum in (28) can be efficiently determined by 
any apfjropriate technique such as the Method of 
Golden Sections fiSJ . ^The- new .grid point thus ere-' 
ated can be ,added t*p the current . set , and a new , 
iteration begins. 
* ' ■ 

Earlier convergence^ proof s [ 10 required that 
all coltimns be retained from iteration to iteration. 
However, recently Murphy [14] haS- developed some ■ 
"column propping procedures^ although no experience 
of the efficacy of , such procedures is cited*-^^ 
. - -ft ' 

It may be mentioned that the constraints In 
problem (27) of the form J Xj^ « 1 can be handled 

t 

as generalized upper bounds. Moreover, the columns 
generated czn be implicitly stored. * ^. ' ' 

IV DISCUSSION, ♦ 

^An optimum integer vector X from solving ^ 
' either (24) or (25) .produces only the menu plan 
whica is to be scheduled by some other, method. It < 
is important to realize that the plan X already 
satisfies the most significant conditions pertain- 
ing to population preferences, total cost, nutri- 
tion,- and other general aspects of feasibility. 
The only remaining objective of scheduling is- to 
assure compatibility among the menu items 'cJithin 
meals and between meals. Compatibility is a prop-*" 
erty of interactions between items, and not' neces- 
sarily a property of the items, per se. Thus, in 
scheduling we deal with acceptability aspects of 
combinations of items ^ which was not directly con— 
sidered in the objective function. The model pro- 
posed tt^us separates the criteria of menu planning 
. and . scheduling into two, distinct optimization pro- 

■ cesses for technical reasons. In the planning J 
pha.se, food preferences are maximized as separable ^ 
functions by the powerful method of nonline^ pro- 
gramming. In '.the schedtiling phase, compatibinty 

is to be maximized by techniques still in the 
exploratory stages. One obvious possibility is the 
arrangement of items by manual raetHods-j More , exact 
approaches are conceivable by' algofi"t&s based on 

■ .TniltiHimensional scaling' arid ^graph theoretical . 
methods presently under investigation'' [4,12]. 

For populations which/are heterogeneous with 
respect to their preferences for a given set of 
menu items, nonselective menus cannot be optimal 
[7]^ An impr9vemei?t on the optlmality cani. bes^ 
effected by partitioning^ the set of individuals M 
into subsets of individuals. Mjj^, (^1,2,***, t) 

■ where t '. ■ ^ 

■ 'U^j^ and each of the t' subsets will 

- 1=1 " ■ ' \i ' / 

be more homogeneous with respect to preferences 
than the set M if the partitioning is done by 
cluster analytic methods [12]. In this case the 
^!G^(X^) function for cluster M^ will have Its- 
optimum Xj values closer to the' individuals' pref- 
erence—quantity function optimums thaif is possible ■ 
for set M . This is simply the theoretical inter-: 
H pretation: af the rationale of . offering selective 
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menus. In^.short^ 'each cluster corresponds '*to a po- 
tential' different optimum, menu plalfx, bence jj^tf^iw 
tlj^ need for ^ cfioice jOn the menif which 'is expect- 
ed^to be exercised by the individuals in the par- 
ticular cluster. By increasing ^^the vafgie of t 
more h9oogea€S3us clusters can be created » more ^ 
individual preferences will coincide with the 
respective Gy. (X^> optiinums^, bu?, 4I so* more 'choices 
■. • . "i.- , ■ ^ 

will be needed aad^'mbre kinds of items are to be • 
prepared^ for each meal- It is conceivable that t, 
i.e., the potential number of distinct choices , ;.has 
some practice bound, and there is evidence [7]. 
■ chat i;he g'reatest improvements in preferences occur 
at low valuers of. t such as t"2 or t-3. 

The^ menu -planning problem for a Ifeterogeneous 
population is therefore pictured as the (joint)' 
.solution of t nonlinear programming problems with 
objective functions 



ters M. 



^i- 



5 functions Gv (X^) correspondl^Vto clus- 
(i-1 , 2 , • • • , tyf yielding optimum non identi- 
cal menu plans for e*ach cluster. THis is to say 
that the planning problem associated with selective 
menus is not particularly different from the 6ne'~ 
described earlier. The scheduling problem becomes, 
however. Somewhat complicated. 

^ * ■ ' ■- 

T-et us' deno^ the cardinality of cluster by 
m^, and the cardinality of M by m^, whe^e 

°c " I consistency between menu plans X^, 

X", X and the- corresponding selective. menu 

schedule requires that -if v^em j is represented 

t 



in the plans in quantities , x^. 



then 



the time averaged marginal probabilities of choos- 
ing these*, items from the schedule in s days must 
be equal to the marginal probability of choosing 
item Xj from other items in the 'given course by the 
total population in . s days, which is equal - on 
the basis of cluster preferences - to 



(29) 



i ^sm^ 



i 

m.x. 

3. j. 



It is a difficult *and thus far unresolved problem 
to schedule selective menus which satisfy this cri-^ 
terionj because compatibility between menu items 
affects the joint probabilities of selections: 
Even if condition (29) is satisfied, the* freedom 
6f chQice provided by selective menus Introdt^ces 
random^variables in the 'foo'd service System, an4 
necessitates redefining the food cost, nutritional, 
and other constraints in. probabilistic t&rms.*' 

.One can, of course," avdid some of these pro^- 
, ^lems by scheduling each of the X^ option pT-ans as 
nonselective menus, offering selectivity only 
among the menus, but noti^ic^e items, *and ^hoping that 
on the average y population" cluster will prefer 
to select .the corresponding, X^ menus 'from the 
schedule.. Obviously; more research is needed on 
■these ^points. ' . *' 

V \ COMP.UTATIONAL RESULTS ,' 

* • "^.^ > . . . . . ■ , 

The grid-line arizifig algor^hm. to solve the 
preference-maximizing menur^lanSing problem was 
coded in FORTKAN for the CDC '6600.' The program 
requires approximately 25000 words of active 



storage t<j solve '30-cbnstraint, 320-variable prob- 
lems > where the number of variables refers to the" 
f number be'foxe line^ization. No passive storage i's 
us^T Ap^proxdLmatj6ry"7000 words of storage, are. ^ . ' 
-t"problem-6l^-independent. Slice integrality of the 
' solution is desired, only' integer grid ipoints are 
5 .used, and t^he-pro^ram-^s-the c a pacity^ ttr-enforce- — 

different: upper bounds on the variables. 
- . \ 1 

Twienty-constraint, 220-varialjle problems re- 
quire ^approximately 20 CPtJ^ seconds, and 20-con- 
straint, 1207variable problems take less^than 10 
CPU seconds on the CDC 6^0^ FurtheiTupre , if a ' 
"goodystarting solution o.^ "used, the ^olution 
times gecrea&e markedly. - ' > - 



^ Portions of the solutions to. two sample prob- 

cLeras' are' presented below. The nutrient and cost ' 
attributes are ^rom a U.S. Army Master Recipe File, 
The problems consist of deternix&ng serving frequen- 
cies of menu items' for a 42-day evening meal 4:ycl^^. 
Although six-course meals were planned^ the display 
contains frequencies of only the- ep trees ""'und^r raw 
food, cost budget limits of $42.00" and $35,00 for 
all the courses. 



NO. 



ITEM NAME 



SERVING FREQUENCY, 
(per 42 days) 
Budset«$35 Bud):^et=$42- 



1 
3 
6 
S 

'9 
11 
15 
22 
23 
f4 
26 

-27 
28 
29 

'30 
3^ 
37 

,40 
42' 
44' 



ROAST :beef . 

GRILLED BEEF STEaSC 
SWISS STEAK W/BRDWC; CRAVY 
MEAT LOAF 
GRILLED SALISBURY STEAK 
SWEDISH MEATBALLS-^ 
BAKED HAK 

GRILLED SAUSAGE P^IES 
BARBECUED RIBS 
ROAST VBfL 

BAiCED CHICKEN ■ - • 
FRIED CHICKEN 
ROAST TURKEY 
HOT TURKEY SANDWICH 
FRIJ!D fish' 
FRENCH fried" . SHRIMP 
SEAFOOD PLATTER . 
■SPAGHETTI W/MEATBALLS 
BEEF STEW 

CHILi CON CARNE W/ BEASTS 



3 ..00 
3,00 
1.00 
2.'00 
3.00 
2.00 

i.oo 

llOO 
- .26 
I.OO 
2.00 
4.. 00 
1.00' 
4-00 
1.00 
2.00 
'2 .-00 
.4.00 
1.00 
3.74 



PREFERENCE 85.24 



4.00 
5.70 
2.00 
2.00 
■ 2.00 
1:00 
1,00 
0 

1.00 
1.00 
2.00 
•4.00 
" 1.00 
3.00 
.30 
2.00 
2.00 
4.00 
1.00 
3.00 

86.69 



In' both solutions, the use of integer grid points 
helps make the solution almost integer, with only 
two fractional values for the variables. We are 
currently using a search procedure to round the 
solution obtained from the npnlinear program by 
permitting fractional variables to change only .to 
the next lower or next higher integers, and non- 
fractional variables to change by at most one. 
Other schemes such as in [1] are also possible. 

Even with a 42~day"* budget reduction of 16Z, 
the solution is able to exploit item substitution 
to yield less than a 2% reduction in acceptability. 
Unlike conventional trial-and-error menu planning, 
procedures, this method prevents over-reaction to 
budgetary , and price f Ijj^tuatdjpns . 

An apparent shprt coming of the procedure is 
that acceptability, (and schedulabillty of the item 
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frequencies obtained from the mac hematical ^^^ram 
depends on putting" together day-to-day combinations 
of irems, that are compatible. The nonlini^ar pro- 
gram^ bypasses ..the issue of compatibility Howev'er, 
bZirtint compatibility effects 'can be incorporated' 
in; ^e form of constraints • For example, i? a 
*&a^wich a^2^^^ tfie menii, there may. 'be no need 
tojs'erve bread in" addition* This situation -can be 
hand33te"a» via tfee constraint 



. * jGS jcB. . 

wl\e^ce K *i«-che number of days in the cycle, B 
is .the index -set^pf breads, and • S is the index 
set of items thatp'prrecliide bread from appearing ^on 
the menu. We enforce such "exclusion" constraints*: 
for, several coiu^es. Another type, of constraint is 
the "inclusion" constraint, which^ enforces the 
appearance of one item if another appears. Thus 
Applesauce can be forced to appear as often as- Pork 
Chops. In our expirieiice, once the constraint set 
considers sch^ulattllity, the compatibility p3?bb-. 
lexn becomes easy to\ handle. 
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Figure 2- The shape of the f(t), h(t) and g<t) 
functions for parameter values of^ ~~ 
a-90.67^ b-67.14, c-0. 0884 and 
*^ r-0.3760- " « 

Source: Reference [6]. 



Subject /Icea 



Codes 


• 


b r 


c 


■r 




V 


1/u 


SO6/5909 


20.00 


10.34 


0-0336 


I.3S08 


1.0 


1.1-iSO 


o.:o7i 


SO6/60I1 


25.00 


11.61 


0.0374 


0.2817 


4.0 • 


1.0000 


0.0200 


SO6/5037 




15.17 


0.0334 


O.KSO 


5.0 


0.S59O 


0.0104 


506/51 11 


30.00 


9.12 


0.0359 


0.-0718 


7.0c 


1.0631 


0.00869 


S07/5082 


80.00 


46.87 


0.0628 


0.1848 


7.0 


O.-^089 


o.oo4o; 


SO7/590» 


90.00 




0.0624 


0.2986 


5.0 


0.91^9 


0.004S6 


SO7/5107 


100.00 


45.78 


0.0805 


0.3442 


3.0 


0.9283 


0.00693 


S07/3013 


•60.50 


21.27 


0.0451 


0.0114* 


30-0 


1.4754 


0.000271 



Figure 3. Estimated parameters of tvb* analytical 
models of the preference-time function 
for foods. \v ^ . . 

Source: Refc^Siee 16]. 




Figure 1. 



The chanj;e of preference over time when 
the item is consume^ on days 9, 16, 19 
and 28, the item having; been consumed , 
9 days prior to day 1. The p.nraneters 
of tHe preference-tine function used for 
this plot are: a=lOO, b=40, c»0,05, 
r»0.4. , 
Source: Reference [?]♦ ^ 
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Figure 4: 



16.00 



20.00 



•QUANTrrY(SERVlNf5S IN 28 DAYS) 

■ ■ * - 

Unconstrained preference-quantity 
functions of an astrdnaut for 'two 
entrees. 



ERIC 



104 



11 



( 



^* ' ^ k REFERENCES ^ 

• . . \ ' ^ ^ . 

£13 Armstrong,. R.'D., a. t. BaXiatfy, P. Sfiiha, "A 
Mttlf;jLstage Scheduling Algorithm for Preference 
' * Maarlmi zed . Selective and Nonselective Menus," 
•^ 0!#^t Technical Jleport No. 7-, School of Business 
" 'AdTTrf pi St rati on University of Massachusetts > 
^ tJuly 1973. ' , " , 

Planning 'bj; Computer . 
. r The Vraypagications >pf ACM . Vol. 7, April 1964, 
pp.25§i759T ' / / 

[33"-_ "^Mathematical Programming ^Sy&tem for 

' Food Majaagement -^plications," Interfaces , 
Vol, 6, Not 1, ?t. 2 (1975) pp^.13t31. 
" [4] ^ , •'Large-Scale- Prograipniing Properties of 

Menu Planning and* Sciifeduling," ±n Optimization 
Methods for ResougcegAllocatiqp. , ed>* R.W. 
<5ottle~and J. Karp English Universities Press 

- Ltd, London, 1976, pp.SJ^-SS^'" 

153 5 J. Neter, W- Wasserman, Experimen- 

. ^ tal Comparison Betwec^ Fixed ^<i^ight ^^d Linear' 
Programming Food Price Indexes , " ^ > Amer ■ Stat . 
• Ass., Vol. 65, No* 329, March 1970, pp;49-60. 

[6] ' , W.J. Duffy, P. Sinha, /"Modeling Food 

Preferences Over Time, ^ Operations Research . 
Vol. 22, No. 4, (1974) pp.711«727- < 
- Sinha, "Computationai^.Prijacip.les qf 

Choicegroup Generation for Selective Menus," 
ONR Technical Report^. 5, School of Business 
Administration , University of Massachusetts, 
' October 1972. • ^ 
' ES] Benson, P.H., "Ps;^chometric Approach to Pre-^ 
■ .dieting Consumer Behavior," Personnel Psy- 
chology ,^ Vol. 13, No. 1^ (1960) pp. 71-50. 
[9] Bowman, J.D. , E.M. Brennan, "Computer Assisted. 
Menu Planning Provides Control of Food Stervice,' 
Hospitals , Vol. 43, No. 103, August 1969, . 
pp. 107-113. 

[10] • Dantzig, ^G..B. , Linear Programming and Extent 
sions, Princeton University Press, 1963, 
^[U]. Gelpi, M.J., J.L. Balintfy, L^C. Dennis, 

■r X.K" Findorff, "Integrated Nutrition and -Food _ 
>r Cost Control by Coaster , " J. .toer . Diet . Ass J , 

- Vol. 61, No. 6, December 1972, pp. 637-646. 
[12] Green, P.E., F.J. Carmone, MultidimensioTif^l 

Scaling and Related Techniques in Marketing 

■ Analysis ,- Allyn and Bacon ^ Boston 1970* ^ * 

[13] Lasdon,'.L.S., Optimization Theory for Large 

Systems , The Macmillan Company, New York 1970. 
[14] Murphy, F.H. , "Column Dropping Procedures for 
the Generalized Programming Algorithm," j 
* Management Science , Vol. 19, No. 11, July 1973, 

pp. 1310-1321. 

[15 J Peterkin, B., USDA-ARS. Personal Conmunication. 
[161 - Smithy Victor E. , Electronic Computation of 

Hrrman Diets , Michigan State University Business 

Studies, East Lansing 1963. 
[17] Stigler, G.J., '"TixB Cost of Subsistence," J. 

Farm Econ . , Vol. 27, No. 2, May^ 1945, pp. 303- 

■314. 

[18] Wolfe, P., "Foundations of Nonlinear Program- 
ming," "in Nonlinear Programming , J. Abadie» ed., 
John Wiley & Sons, Inc., New York 1967. 



ON THE ANA,X,ysiS AND COMPARISON Of 
-MATHEMATICAL PRO.GRAMMING ALGORITHMS 
^ AND SOFTWARE 

Ron S. Dembo 
YaJLe University 

' ■ John M, Mulvey 
Harvard, University 



' ^ Abstract 

Although mathematical pro^rairnning al- ^ 
gorithms and related con^uter software 
i^ave-'been in existence for over twenty- 
five years, and new methods are being in- ■ 
vented, revised and implemented at a rapid 
pace^ there have been f^ew^ (if argT con- 
' - Crete suggestions for conducting experi- 
ments to evaluate competing numerical tech- • 
nigues. Yet -the ^samputer- ope rat ions re- 
search folklore' abounds ' with information 
^ about the reputed eff iciencies . of. various 

progrctms. Ij?^n initial attempt at improv- 
. ing this condition , we analyze the problem 
from a statistical point of view.,-. The ap- 
proach is contingent on the existence of a 
well-defined population of test problems 
, from, which a statistical sampling is car- 
ried out- By measuring the relative per- 
formance of various, codes on the sample 
problems, predictions can be made as to 
■ their relative** performance on the popula- \ 
tion of problems under consideration. Fur- 
thermore, the significance of these predic-- 
tions can be measured in a rigorous way us- 
. ing standard statistical procedures. As a . 
demonstration of our methodology we con- 
sider in detail a comparison^of a primal- 
simplex network code with one based on ^e 
out-of-sikalter method. We show how one may \ 
define^ in a precise manner., .a population ) 
of test probl^s and what conclusions . may 
be drawn from a simple random sampling 
procedure . • . 

I. Introduction 

]■..■•'■ ■ 
. Sinjce the inception of the simplex 

method for linear programming thirty years 
.ago and the simultaneous development of 
^computers, researchers have been concerned 
Vith analyses and comparisons .of mathe^- 
inatical programming techniques. At first, - 
variants' of the simplex method such as the 
revised simplex method, and pro dutt form 
of the inverse, were proposed as alterna- 
tives to the original simplex design and 
later as inteimal routines* for a variety of 
nonlinecir and combinatorial- progrcunming 
techniques. One of the first empirical an- 
alysis of these proposals was by Wolfe and 
Cutler [1963J* A plethora of compu tat ionalV 



studies have followed (cf • Kuhn and Quandt 
[1963]/ Florian and Klein . 41970] , .Gilsinn 
and Witzgall [1973], Srinivasan and Thomp- 
son [1973], Zanakis [1973] and Barr et al 
[1974] ).• Paralleling these studies have 
been a host of informal unp;ablished exper- 
iences from which, "a sxibstantial .folklore 
about the 'relative effectiveness of vari- 
ous techniques has arisen; despite these 
studies, 'there i^ still little, agreement 
today.. Since the out'^of-kilter network al- 
gorithm (a primaO^^dual approach) was pxib- 
lished in Ford and Fulkerson (1962], for 
example, a debate has raged over the rela- 
tive superiority of this method -versus the 
network^specialized primal simplex methods 
Csee 'Dantzig . [1963^ , Aashtiani [1976], 
Klingman et al [1974], and Hatch [1975]). 



There are several underlying causes for 
this lack of agreement: {X) the absence 
of a graded set of standard test problems.,. 
(2) uncertainties about the efficiencies _ 
of different computers, (3) incomplete des- 
criptions of the experimental design vari- 
ables when reporting computational exper- 
• ience, and (4) a lack of guidelines, for 
performing computational experiments* 

In other areas of mathematical program- 
ming such- as nonlinear programming, at- * 
tempts have been made to compare -algor- ^ 
ithms, see for example Colville [1970] ^ 
Dembo, [1975] anc3 Rijckaert [1975]:- Here, 
it is much more difficult to come; to any 
firm conclusions regarding the relative 
performance of the particular codes in ^ 
question than in the case of (say) network 
algorithms-.. Factors such as internal tol- 
ercirice settings, accuracy of the solution ■ 
and whether or not a code actually com- 
puted a Kulin-Tucker point, all play a cru- 
cial role in e;yaluating the behavior "of a 
coded alborithm. 

'There is no 'question a^ to the need 
for evaluating the relative performance of 
coded algorithm^. From a theoretical 
point of view, algorithms are often evalu- 
^ ated on a. ."worst case" basis . • This ^ type ; ' 
of « analyses is often misleading in a coded 
form of the- algorithm. For example, it is 
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widely accepted iff^the mathematical pro- 
gramming literature- that Kelly's cutting 
plane algorithm [1960] is not a good 
method for solving convex programming proS*-^ ■ 
lems- in 'the sense .of convergence rates ^nd ' 
numerical' "Stability . It is probably be- 
cause of this fo lklor e that one hardly ever ^ 
.hears o^ nonlinear codes based on Kelly's 
cutting Iplame ,al;g6iuthm« It is qui te , con-- 
ceivable, however^ /^hat such a method^mighrt " 
provide -a basis ^r^a relatively ef f i<iient - 
"and roi^usf code for certain useful classes'^ 
of mathematical programming problems.. This 
has actual.ly been demonstrated recently for 
small- to medium-sized- geometric program- 
ming problems • In two independently con- 
ducted comparative GP studies CBembo [1975] 
Rijckaert [1976]) / a cutting plane. algor- 
i^hm was shown, to be one of the most ef fic- 

•fient^ and robust^ codes tested. 
/ ■ ■ 

One of the major obstacles in conduct- 
ing computational analyses involves concep- 
tual differences between mathematical al- ' 
gorithms/ the computer spftwaire and the de- 
tailed, .problem-specific tactics which oc^ 
-cur when empirical results are collected. 
Figure 1 graphically depi-cts the situation. 
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Mathematical algorithms occur at the high- 
est level* of abstraction (level 1) ; follow- 
ing Zangwill [1969-], we define algorithm Is 
a sequence of point-to set mappings from 
which theoretical results can be deri^^d - 
for exauuple, infinite convergence proper- 
ties, or algorithmic efficiency as defined 
by the number of iterations in a worse 
case analysis. Moving down to the next 
level of detail, we encounter computer' 
software. For each mathematical algorithm 

^In terms^ of standardized central process- 
ing time. 



. flevel 1), there -are a host of computer 
software implementations (level 2) with - 
widely var.ying degrees of e'f f ectiveness. 
These implementations rS^nge* from under-. 
' gra<3uate student' LP pro-jects to- IBM's.MPSX 
systenu It is* important to note that a 
basic, and dis'tin^ishing property of level 
2 is the. underlying information structure. 

For each^^^compiiter software implementa- 
tion, there are usually many contirbl set- 
tings vh'ich define interiial' tactics that 
the program utilizes in solving a . particu- 
, lar problem or a problem-class-. These con- 
trol, settings can result in drastic dif- 
ferences in efficiency, especially for 
large-scale mathematical programs. To il- 
lustrate the variation which "can occur, a 
single assignment network with 10,0bO 
nodes (constraints) and 30,000 arcs (vari- 
ables) was solved (see Mulvey [1975]) with 
three different pivot strategies, that is, 
the procedure for selecting which eligible 
non-basic variable enters the basic at each 
pivot. The computational results are 'as- , 
follows : 



Pxvot Strategy 
(1) 
(2) 

, (3) 



Seconds 
3076 
3028 
971 



Pivots 
472,999 
. 28,113 
^^^6,204 



A 'similar^^'henomena occurs in nonlinear 
programs wtiefe a small change in the tol- 
erances causes -large shifts in computa— 
tional results (see for example Dembo 
[1975]). 

Admittedly, the elements at each level 
(mathematical algorithm, software implemen- 
tation, internal tactics) cannot be pre- 
cisely defined and universally accepted. . 
.To some , a minor change in internal tactics 
is really a. change in the fundetmental math- 
ematical algorithm. To avoid these diffi- 
cult issues, we took a different approach 
for developing a framework with which we 
could analyze and compare mathematical al- 
gorithms and related software . 

In the next section, -^we review the 
various types pf iest pr^dblems that may be 
used * in numer icari comparisons , and Section - 
3 provides the above merttaoned framework by 
concentrating on well-defined collections 
of test problems. A statistical analysis 
is then undertaken in Section 4, In thesfe 
experiments, it is not our primary inten- 
tion to "exhaustively test specific tech- 
niques, but to show how these comparisons 
can be conducted in li,ght of a statistical 
analysis. 



In terms of the number of problems for 
which the ^qode actually, computed a correct 
solution. 
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2, Test Problems r 

There ^re two main categories of prob-;^' 
lems thft are currently used for reporting 
numerical results in the math^atical^ pro- ^ 
gramming literature." Problems a^e* either 
hand-selected or are randomly generated 
using a psteudo-random number generator ..In 
both cases one encounters serious draw- 
backs when attempting a scientific analy- 
sis of computer codes used to. solve' a par- 
ticular class ^ of problems, ^ " . 

On the one hand, the beliavior of com- 
puter codes on randomly generated problems . 
does not in general reflect the behavior\ 
of these same codeis on similar sized real 
world problems. This may be due in part to, 
the fact that in real world problems there, 
is usually son^e degree' of correlation 
among variables. Correlation may be in- 
corporated in randomly generated problems 
to accurately model real world behavior; 
however, both the nature of. the applica- 
tion and. the degree of correlation among 
variables would have to be known and thus- _ 
the problem generator would have to vary 
from application to application. ' » ^ ' 

On. the other hand^ if used for com- - 
parative purposes, comp\itational results 
obtained from problems that have -been - 
selected from mbdelsof real-world situa- . 
tions make a statistical analysis of re- 
sults questionable. In £his -case, conclu- 
■sive statements regarding the performance 
of these codes can only ^ be made for the 
particular problem set under considera- 
tion and any generalizations must be 
treated with suspicion. 

The relative merits of the above 
categories of test problems are summa- 
rized below:' , 



Hand-Sclcctod Probisr.s 

Usually representative 
or rcel-'*'orld behavior. 

Expensive to colle^t^ 
document . ^nd send from 
ortc rcsc:ircher . to 
another. 

Population or problems 
from which sa.T.ple pro- 
blems Arc^ draTwTi is not 
known. Thus, general- 
izations based on the 
sample "aPfcA qucscion- ' 
able. 



Kaiidor:ly Generated ?roblemg 

Usually' not representative 
of real-world behavior. 

Prcblen generators C2m be 
designee' to be portable 
and raachlr.c independent. 

Population of* problems is 
hnown and can be cent rolled. 
If sampling n-.ethod is known, 
generalisations based on 
saniple statistics can be 
made ^»fii:h a /:i;iOwn degree oc 
ccttainty. ^ 



In this study we will restrict our- 
selves to pseudo-randomly generated prob- 
lems'. We have chosen to do so because in- 
ferences can be made about populations of 
test ' problems in a precise manne^r,- 

.Since this is an initial attempt in a 
field that is relatively^ untouched/ we 
have decided, to restrict ~bur attention to* 



comparing th4' performance of twb network 
codes on* a well-defined population pf 
assignment ^problems. Part of our aim will ' 
be to set out*'on defining a standard format 
for researchers, to follow when reporting 
r^esults on the behavior- of coded ' 
^ algorithms* * ^ , : ■ 

The two codes -considered in the • next 
section are: • ■ ^ ' . 

KILTER : A network code. based on an out- 
c of-kilter algori^thin and written 
* by Aa^htiani [1976] 

and ^' • 

LPNE^T :. A- network code based on the. ^l- 
pr-imal simplex method ar\d-,wrirt- 
ten by Mulve^ [1975] ' 

.' ' ' e - ■ 

There * are a number of reasons for 
starting our analysis with network codes as 
applied to the. solution of .assignment prob- 
lems. First, network calculations involve 
\ manipulation of integers and therefore tol- 

\ erances which are difficult to standardize • 
'.and which 'play such an important role in 

■comparing nonlinear programming algorithms,' 
can be avoided. Secondly, a widely used 
pseudo-random problem generator, NETGEN 
(Klingman, Napier, Stutz [1974] ■),• is avail- 
able for constructing feasible' network 
problems. Finally, as previously des- . 
cribed, the recehlf* literature on network 
codes "has been filled with controversy as 
to whether either"' the out-of-kilter or the 
primal simplex method- are the- best ap- 
proaches' to solving assignment . problems., 

3- Notation -and Methodology 

The* primary aim of computational com- 
parisons 4-S to make inferences about the 
relative, behavior- of the various algor- 
ithms uunder consideration. It is widely 
recognized that such comparisons cannot 
lead to' any hard conclusions regarding al-r 
gorithms ■ themselves ; rather, one can only 
derive information on the relative per-, 
formance -of tKe software implementations of 
these algorithms. What is not realized in 
most cases is that the problem of comparing, 
software per^rmance is a statist.i'cal one. 
Namely, the behavior of a.nuinber of comput- 
er codes oil a specially -selected set of 
problems is measured in terms of certain . 
performance indicators and from this^ data 
\ * inferences are made about the behavior of 
these codes on^ a. larger class of problems. 
This is clearly a case of statistica^l 
sampling and in order for these inferences 
to have any firm basis, an -experimental de- ■ 
sign should be carefully thought out wi.th a- 
view to the nature of the inferences that . 
are t6 be made. Such a decision must ^ 

(i) --identify the population from which 
sampling is to take place, 

(ii) describe the statistical sampling 
method, and 
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. , (iii) st^te the hypotheses" that are to 
be tested. 

c. ' , - , \ 

If the above three factors are care- 
fully developed, then once the computer 
runs havev been made and the appropriate, 
variable measured, inferences can be drawn 
-as-to^ the ;relcbtive— behavior-o^f— t^ese^-<:odes^ 
on the population of problems identified 
,in (i)*. Furthermore/ the significance of 
'these inferences can be accurately meas- 
ured using well known statistical tech- 
niques. In order to demonstrate the meth- 
odology, we perform, in detail, a statis- 
tical comparison of the network codes 
KILTER (Aashtianr [1976] J and LP13ET (MUl- 
vey'(1976]). ^ ■ - 

.p In order to describe a general frame- 
work we need to define the following sets. 

P ts the set of aZl ^ problem's that the 
mathemat'ioaZ algorithm upon which the 
code is based, is theoretically cap-- 
able of solving^ 

* 

For example, P (KILTER) is the collection 
of all transshipment problems solvable by 
the out-of-kilter algorithm and P(LPNET} 
isjthe set of all. transshipment problems 
solvable by network specialized primal sim- 
plex algorithms. 

» ■ 

is the s€t of all problems for ./ 
which th^- particular code was designed. 

-For 'e:xamp3,e, P^ (KILTER) is the set of all 
transshipment problems with less thai/2p00 
arcs and 500 nodes and P^CLPNET) is 'E:he 
set of" all transshipment problems with 
less than 2000 arcs and 500 nodes. 

^ » 
?^ is the population of test problems 
and is a subset of P where I 

^ iel 

is the set of codes under mvestvga- 
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P^ is taken to be the 



In our case, 
set of all feasible assignment problems 
as generated by NETGEN (Klingman, Napier 
Stutz [1974] ) , with the following 
characteristics:/ 

, .jiumber of nodes between 200 and - 
^ 500, .■ ' 

. number of arcs between -1000 and 
200, and - 

.range of cost coefficients between 
a lower bound of 1 and an upper 
' bound greater than 100 but not ex- 

ceeding 5000. 



The nature of the population and exactly 
what it represen4:s is not addressed here 
and is a subject for' future reseairch. 



Here, P^ was chos^ because of the physical 
limitatio'ns of the computer used,-"- our 
self-imposed restriction that the runs ^ 
should be carried out internally,"^ and- the 
•desire lio compare these codes with respect 
to sparse, small-scale assignjpsent proi>lems. 
Table I below gives the details of the 
-storage— Uiigh-speed memory) requirements 
and other important factors that a2:e re- 
quired for reprodiacability of our experi- 
ment. 

CODS bi»mx ncAgiows 



Compiler 
Pr«cisioa 



P (1CTLTSR) 

^371: Word* 
. sranrtard rorCrmn 

DEC 1070 CHBS^) 
. Fortr«n-F40 



P f LPSET') « 

16K Words 
St^adard Fortran 
cfec J.070 (HBS) 
5i^rt^kn-F4 0 
Inc«g«r 



A typical sampl^g procedure would be 
the following. 'Draw, a simple .random ■ sam- 
ple, p, of n problems ,.from the test prob*- 
lem population P^ - The size of n is 
chosen to 'be large enough so as to ensure, 
for exaimple,' that the distribution of 
sample means (of *the , particular per:^orm- 
ance-measure, e.g., run time) is normal. 
■Methods, for choosing n are described in 
any elementary statistics text. A decid- 
ing factor in the choice of n might be a 
limit ^on the" acceptable pi;pbability of 
rejecting an hypothesis that. is actually 
true* (Type I error)*. 

In our case the 'simple random sample 
was chosen with the aid of NETGEN, a * • 
feasible pseudo-random network problem 
generator developed by Klingman, Napier 
and Stutz [1974]^.' 'This choice was pri- 
marily giiided by a desire to* make our 
experiment easily reproducable; NETGEN 
serves this purpose since it may be 
rea'dily obtained from its authors and is 
already widely used. Details of our sim- 
ple random sampling procedure are given 
below. 



All testipg was performed at Harvard Uni- 
versity on the DEC 1070 computer with a* 
maximum, 'of 64K word's of high-speed memory. 

2 " 

.VJithout resorting to auxiliary memory* 

i3 -i - 

'Harvard Business School. ^ ' _ 

A 

The authors are willing to perform iden- 
fical experiments with other generators 
provided . the I/O' formats conform to 1;he . 
SHARE convention and the generating pro- 
gram can be successfully compiled on the 
DEC 1070 cojjiputer (FORTRAN) , * ■ 
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:ed'?T^e for Selecting .a Simple Random 
S^T"ple of Test Problems from the , 
i^opulation " 



Step 1 , Generate 5a random integer, in 
7 ti^e~range 100 to 5OO0- This fixes the" 
range of cost coeffic'ienti^ to the 
range 1 to d. ^. *■ 

Step »2_ , Generate a random integer, 
between. 1000^ and^ 2000', This fix^s 
, the number of ,arcs- (yariable^s) to n^--^ 

Step 3 . Generate a random integer-/ m , 
between 200 amd 500. -This f ixes^ ,th'e 
number 'of nodes (constafaints) . 

Step 4 . * Given the problem, parameters , 
specified in Steps IV 2 and 3 use , ^ 
NETGB^r toS^enerate a single " feasible , 
assignment problem with n^^ variable!" 
* (^Step 2)" and m , constraints^ (Step, 3) 
2Lnd ,cost coefficients, randbrily selected 
from a uniform distribution in the 
range £o.d. 



Step 5- ^Repeat steps l/"2, 3 and 4 until 
the desired' scimple size has been 
reached.' - . 

^ we should note here th2it 'we assume 
'that the mamber of arcs. and_the number of 
nodes .are . uniformly distributed in the 
ranges 1000 to 20.00 aiid 200'to 500 "re- 
spectively. ^ ' ' 

In ordeiyto perform the statistical 
analysis in Section 4, we selected two 
independent sim^e random samples each^ - 
containing 50 problems. The problem sets 
are' all the necessairy information re- 
. quired to reproduce them are given in 
Tables 2 and '3 below* The optimal . so lu- ^ 
tion-s as well as the 'run tomes ' 
required by LPNET- and KILTER are given in 
Tables 4 and 5 - ^ ■ 

In the following section we perform 
.a statist.ic.al' analysis of the results. 



FlOBLSl SPECnXCAZZOHS lOK FISSX TOTZ IZST CASES 



Tables 

YlOtIZM SracmCATIOKS jot StCOg) TUTTt CASZS 







■ ■ 


Co«t Co- ' 






OBS 
1 


f 30dm 


■ 9 Arcs 






^ Smmd 


284 


1426 


2176 


.071 


-.284 


2 < 


422 


1376 


4394 . 


.031 


'422 


3 ' 


466 


1160 


4930 „ 


.021 


466 


S - 


488 


1776- 


4086- 


.030 


488 » 


5 


234 


1985 


. 1041 


.145 


234 


6 


430 


1447 


2859 


.031 ^ 


430 


7 * ' 


474 


1082- 


1136 


*019 


474 


Z 


434 


I. 1217 


339 


.026 


434 ' 


9. 


304 


1628 


2144 


*C70 


304 


10 


486 


1080' 


2081 


.018 


486 ' 


11 


324 


1669 


-4^ 


.064 


324 


12 


462 


1395 


2553 


'.026 


462- 


13 


* 398 


1353 


^ 2348 


.934 


398 




456 


1160* 


150 ' 


'J022 . 


456 


15 


268 


1188 


4421 


.066 


268 


16 


4X6 


1813 


'689 * 


.042 


416 


17 




1579 


3567 » 


.081 


280 


IB 


282 


1663 


3185- ' 


-084.' 


282 


19 


300* 


1775 


1653. , 


.079 


' 300 


20. 


396 


1755- 


3987 • 


.045 


396 ■ 


21 


438 




4933 • 


.023 


438 . 


22 


296 


1504 


4444 ^ 


.069 


296 


23 


240 


1722 


2066 


.120 


240 , . 


24 . 


496 


1609 


,1830 - 


.026 , 


496 


25 


• 494 


1897 


^2708 


.031 


494 


26 


' 268 


1534 


4235 ^ 


.085 


268 


27 


296 


1870 


3765 


.085 


296- 


2S 


340 


1927 


2603 


.067. 


340> 


29 * 


340 


1731 


3087 


..060 


340 


30 


348 


94a 


. 4653 


•045 


348 


31 


402- 


.1880 


4447 


:047 


402 


32 


242 


' 1187 


2394 


.081 


242 


33. 


378 


1672 


1659 


•.'047 


j378 


34. 


216 


1706 


4320 


.146 


216 




382 


1497 


2771 


.041 


382 


36 ^ 


290 


1309 


233 ' 


.062 


290 


37 


. 494 


1944 


386S 




494 


3S 


460 


1097 . 


3592 


460 


39 


482 


1541 


2837 n 


.027 


482 ' 


40 


.232 


. 1260 


3473 


.-094- 


232 


41 


454 


1951 


2522 


-038 ■ 


454^ 


42 


224 


1493 


3985 


.119 


224 


43 


386 


1560 


891 


.042 


386 . 


44 


364 


1448 


4823 


.044 * 


564 , 


45 


380 


1833 


4137 


.051 - 


380 , 


44 


242 


1471 
•1836 


4809 


.100 


42'4 


47 > 


430 


i:689 
"4176 


.040 


430 


48 


202 


.1466 


.144. 


202 


49 


304 


1157 


1746 


.050/ 


304 


30 


278 


/ 1240 


1980 • 


,064 


273 





# Area 


*rflel«ne 








442 


1716 


4200 


.035 


442 


■ - 52 


208 


1611 


2378 - 


M49' 


208 


' 53 


420 . 


1293 


. 1063 ^ 
.4233 ^ 


.029 


420 


5* 


3W 


1428 


.045 


358 


55 


26* 


1963 


. ^ 3811- 


.111 


266 




402 . 


1673 


1601 ' 


■ .041 


. 402 


57 


466 


lOlJ 


666 


.019 


466 


5B 


402 


1993 


860 


.049 , 


402 


59 


342 . 


1063 


2493 


.036 (f 


342 




348 


. 1398 


2343 - 




.348 




462 


X966 


792 ' 




462 




362 > 


1769v 


3152 


.054 


362 


. ' i3 


47ft 


l&OO . 


3412 


.>032- 


478 ^ 


* 64 


216 


1689 


U76 


.145 


216* 


,~ 65 


332 


1734 . 


4280-. 


_U033 


352 


66 


294 


1283 


3346 


.039- 


294 


67 


258 


1479 


' 1279 


.089 


258 


' 6« 


234 


1747 


*1244 


.128 


234 


69' 


356 


1469 


1999 , 
3520/' V 


/ .046 


356 


70 


292; 


1798^ 


.084 


292 


= ■ 71 " 


330 


1180' 


ti5o - ^; 


.043 


330 


■72 


274 


1076 


1769 % 


^ ,057 


' 274 


73 


422 


1464 


2241 - S 


...033 


- 422 


74 


432 


1168 


1704 I 


I .023 


432 


73 


326 > 


1330 


3O03«-l J ; 


\ c058 


326 


7* 


236 


1676 




\.120 


236 


77 


430 


" 1267 


■ 

3906 \ : A 
, . 122 


.C25 


430 




356 


1813 




356 


79 


246 


1450 


: !^ 


246 


80 


- 238 


•1225 . 


-2543 - 




' 238 


81 


336 


U02 


3719 ^. ■ 




336 


62 


'216 


1205 


1239 " 


. ao3 :^ 


• 216 


' 83 


,488 


1336 


3201 


*026 


488 


84 


-468 


1863 


128 


*034 ^ 


468 


85 


244 


. 1119 . 


1751 


.07S. 


^ 244- 


86 


•434 


1550 


45U 


.030 ' 


7 *34 


- 87 


308 


1537 


, 4549 


,063 


V.-308 
^ • 486 


88 


486 


1499 


4375 


' ,025* • 




442 


1698 


1476 


.035 . 


'<**^ 


- " ■ 90 


232 


1415 


2396 


,105. 


^232 


n 


230 


1307 


3163 ' 


■ .084' 


'vp 250 




228 


U58 


4336 




■V'^ ' ■ 


93 


372 


1363 


" 3278. 




- - .^72 \ -V' 


94 - 


236 


1468 


181 ' 




" '',256 


95 


292 


1508 


734 


.071 ^r. 


. \292 


96 


496 


1583 


4176 


.026 V 




97 


420' 


1114 


. 3781 ' 


.025' i 


■ W 


98 


39* 


1020 


3388 


.026 


V 398 


■■ 99- 


•382 


1207 


3049 


.033 ■■ . 


382= 


100 . 


424 


1657 


4443 


.037 


424 



110 



22 



Table 5 



Tabic 4 



^ . 'rON time, results for FIEST 50 PROBLEMS 
(DEC 1070 Seconds) 









Qprinax uojeccxve 


OBS 
1 


KILTER 


LPNET 


Functii.on Value 


10.700 


3.1^00 


54393 


2 


15.900 =- 


6.800 . 


\ ^32241 


. 3 


15.300 


7.100 


354043 




17.600 ' 


9*400 


235820 


* 5 " 


11-760 ' 


4:600 


170^7 


6 


13.900 


6.400 


141765 


7 


7.200 


4.500 


87847 




16.100 


£.500 


36739 


9 


11.600 


4.1O0 


59286. ; 


10 


13.600 


G<.900 


171024 


U 


16.300. 


.'5.200 


132492 


12 


20.600 


6,300 


175529 


- 13 


. V46o 


.5.700 


' 117919 


14 


8:700 


6.000 


10228 


15 


8.100 


^ 3.100 


128061 


16 


23.200 


7.100 


' 30992 


17 


12.100 


3.900 


89323 


18 


16.000 


3.900 


77548 


19 


13.000 - 


5-100 


43025 


20. 


19.500 


7.100 


.154219 


21 


10.200 


4.200 


329131 


22 


13.200 


4.600 


109068 


23 


12.300 


3.100 


37046 


24 


24.300 


7.900 


118056 


25 


21.700 


9.200 


143626 


26 


' 9.300 


3.800 


97023 : 


27 


17.700 


5.100. 


90408 


28 


18.900 


5.100 


18911 


29 


13.400 


5.000 


' 94035 


30 


14.300 


4: 100 


190304 


31 : 


21.200 


8.000 


175347 


32 


6.90q • 


2.300 


56866 


33 • 


15.300 


7.70a 
3.300-^ 


67607 


34 


14.600 


69186, 




16.100 


4.900 


108440 


36 ^ 


5.200 


4.900 ; 


6944 


37 ' 


^-^200 


7.500 


213927 


38 ' ' 


10.600 


5.400 


260746 


39 


19.300 


7.100 • 


160290 


40 


6.300 


3.300 


75499 


41 


27.100 


8.800 


123985 


- 42 


12.600 


3.100 - 


70750 


43 


18.700 


6.400 


38006 


44 


12.800 


5.600 ^ 


190868 


45 


21.800 


5.500 


'^'158790 ' 


46 


13.000 


3.600 


110732 


47 * 


22.700 


7.300 


81686 


48 


8.800 


.3.000 


^ 39916 


49 


10.200 


4.100 


64907 


50 


10.700' 


4.100 


57996 



RON TIME RESULTS lOR SECONO FIFTY PROBLEMS . 
' (DEC 1070 seconds) 



Optlxsal^ 
Objective 
Function 







Value 


51 


6.950 • 


218462 


52 


3.940 


37489 ^ 


53 


6.500 


' 61535 


. - 54' 


4.260 


168882 


55 


3.690 


72642 


' 56 


7.890 


- \ 67737 


, 57 


5.700 


55139 


58 


7.250 


3325S ^ 


59 


4.090: 


^ 118324 


60 


6.060 


. ' 86894 ^ 


61 


7.800 


41309 


62 


6.470 


103369 ' 


63 


8:220 


188851. 


64 


4^.170 


17434. 


65 


5.990 


144286 


- 66 


. 3.390 


99763 


67 


3.400 


32124 


68 


4.000 


23338 


69 


5.810r 


83422 


70 


5.040 


85787 


71 


4.600'' 


66249 r 


72 


3.470 


. 57980 • 


73 


7.470 


109052 


74 


4.670 


115475 


75* 


5.950 


97911 


76 


4.260 • 


75170 


77 


6.520 


130922 


^ 78 


7.000 


131819 


79 


3.600 


2806 


80 


3.8ia 


56397 


81 


4.100 


. 172230 


82 


2.960 • : 


"23385* 


83 


8.600 


208613 


84 


8.680 * 


6732 


85 


3.320 


46686 


86 


8.710 


253162 


87 


4.310 < 


139037 


88 


7.860 - '> 


30102 


89 


8.250 


80863 


90 


f3.500 


52679 ' 


91 


3.100 


72179 


92 


3.640 


89093 


93 


5.590 


148160 


94 


3.310 


4304 


95 


5.110 


19390 


96 


7.200 


263007 


97 


5.400 


'226245 


98 


4.570 


• 20U28 


99 


5:650 


153775 


100 


7.140 


199224 
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4* A Statistical Comparison of As^gjri- 
ment Codes 

Summary Stat is£ics for the sample 
- 1 sets are given in Tables 6 amd 7 • r 



sxaoaax statistics for first nmi pjioblems 

Max jtoM 

•496 ^ 360.04 

1985 1526.96 

4933 2944*36 
.146 .058 



. Kin 
^02 

Arcs ^ 1080 
CP«t 150 
OmxxMLtY .018 



89.98 

'.267>5 

1438 
.0337 . 



T«bl» 7 



SnWHAgy STATISTICS POR SeCDUD yiTTY PHOBLEHS 



. / Kin 
208 

Arc* 1080 
Cost 122 
OsBsity .019 



Max 
496 
1993 
4549 
.149 



Mean 
350.2 

1479 13. 

2584.06 
*059 



Std.Dev. 
88.56 
264.7 

, 1330 
.0337 



Notice the ^relatively large standard dev- 
iations for the number of 'nodes, and arcs, 
and the cost . range ; this is due to the 
usage of a uniform density f unctiorl in 
generating problems. Since the" problems 
within p are' "small-scale" .examples, we 
wanted the chauracteri sties of the prob- 
lems wisthin p to be selected' with equal 
likelihood* For this reason a uniform * 
density function was employed. 

Turning to the computational results - 
displayed in Table 4, it is interesting.^ 
to observe that , although KILTER is f xiLly 
2.7 times ^slower tiian LPliCT.'bn. the aveor- ^ 
age, there is some varicibilxty* Specif- 
ically,. KILTER is only 1. 45 times, slower - 
than LPNET for. problem #14, and a similar 
Occxarauice'^is .demonstrated for problem #36. 
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It is in^teresting * to observe thE^t 
LPNET and KILTEI? are moderately corre- 
-lated among themselves with a positive 



correlation coefficient of .7286, and 
'that their relative dispersions (means/ 
standard deviations) are approximately 
equal (see Table 9). 

- The. next set of experiments tested 
LPNET on problems 51 through 100. Notice 
in Table 9 that the sxammary statistics, 
that- is, the mean, run time, standard 
deviation and s/x ratio for the experi- 
ment, are approximately-^ equal to 'the 
summary statistics for the first 50 prob-r 
lems. Since this set is independent of 



Table 9 



RUN 


TIME 


SUMMARY, 


STATISTICS 














KILTER 


LPNET 


First 50 


X 


= 14.698 


" X = 5.426 


Problems 


s 


= 5.223 


s = 1.730 




s/x 


= .355 ' 


s/x = 0.323 


Second JSC- 
Problems 






X. = 5. '459 
s^= 1.743 
s/x = 0.319 



the first 50 problems, we are able to 
examine th^ distribution of sample means 
of size 50 for LPNET- run time. -Because 
of the relatively large , sample size, 50, 
the .distribution of sample means is nor- 
mal, regardless of the nature of the dis- 
tril^ition of individual run times. The 
sample mean is, 5. 459 which ^s a poii^ * 
estimate of the population mean run £ime* 
The estimated standard error of the^mean 
is 0.249^. The small standard deviation 
is a result of the relatively' large num- 
ber of a observations in the sample p. In 
a similar fashion, a point estimate of 
the mean run time for KILTER is 14^7 and 
an estimate of the standard error of the 
;neah. is 0.75. 

The di'stributjLon of. .sample mecins may 
also- be used to generate confidence int^- 
vals on the meam CPU execution times for 

"^IThere/are many -useful tests that may be 
executed with data obtained from two in- 
dependent simplfe random samples.* . For 
example ,^ inferences made using the first 
sample^ifey be .checked usitig hypothesis 
testing based on statistics measured on 
the second samplie. 

2; 



Standard efror'^^^^ the 



mean a = s/ Vn-l . 
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124 



. - • ■■[ ■ ■ ■ . 

the population ofproblems , Pt # ' for both 
KILTER and XPNETl^SSf we denot-e by PK' - 
sind ux* mean CPU time in seconds for 

KIliTER and LPNET for the jjopulation Pt,^ 
we can construct** the following 95S.con-^ 
f idence intervals : 



13,23S < 



16 A62 
5.9X6 



7,704 ^ Uj^ - <. ^0 



'Since CPU times can"* be ,4nf lu'enced 
by the number -and type of other programs 
which were operating during execution, we 
carried out the following experiment to 
measure the extent of the variability 
which can occur in the DEC-1070 environ- 
ment at Harvard. A single test problem 
(456 nodes, 1512 arcs, cost, range 
[1,1414], seed 228) was solved by7LPiaET- 
at seven randomly selected times during a 
one week period- The resulting CPU times, 



are: 








1) 


8.4 00 


seconds 




. . 2) 


8.331 


n 




3) 


8.933 


tt 


* t fully occupied).. 


4) 


8.210 


tt 




5) 


7.966 






6) 


7.919. 


tt 


{ ^ empty) . 


7) 


8.,J575 


tt 





Thus . from the above data we can expect 
the maximum ^r^or' in measured CPU time 
to be of the order of 13%. . It should 
* be noted J:hat these variations are 
caused by Stutomatic. "swapp»g" of jobs 
between central memory and^auxiliary 
stdraga^. - This phenomenon/occurs when- 
ever a ^c3mputer installation n^r forms 
in a multiprogramming environment. ' ^ 

A linear regression analysis based • 
on the first sample of *50' problems^ was ^ 
undertaken to determine the^pverall com- 
bined effects of problem specifications 
and performance, as measurecj by CPU. time. 
Several regression models were tried* 
' The most appropriate linear regression 
equation for LPNET was found to be : ^ 

y/LPNET= Bp 



^2^2 



where 

y/LPNET= run time for LPNET in seconds 
Xj^ = nxamber of nodes . 

X2 = number of arcs 
and the estimated coefficients are / 
given by . - 

B« = -4.393 with a standard deviation of- • 

' .\ ' -"^^ ■- ■ 

6, = 0.01654 with a standard deviation o'£ 

^ ^ - .00120 - , ' - 

... » ' 

g = 0.00255 with a standard deviatrion of 
' , . 00038 . . - 

■ ' The . estimated standard deviation of 
the residual for the a above regression 



equation was ..72 and .the coefficient of 
jdet elimination (sample* R^) was computed to 
be" oT^45. The relatively* small standard 
deviations of the sample regression coef- 
ficients BOf Si and B2 indicate tjiat the 
sample estimates Sor Bi and B2 ^^re &ig- 
■•nificant. Another way of saying this 
would be that the CPU run time o*f LPNET 
depends to a signi'^icant extent on the 
ntimber of nodes and number, of arcs in the 
problem being- solved. The high r2 (.84 5) 
means that 84.5% of the variance can be 
explained by the size of the problem as 
measured by the number of nodes and arcs. 

Although the relationships are ap- 
proximately linear over this collection 
of problems , we do not expect that a 
strictly linear extrapolation can be made* 
-to larger problems. , Caution should be 
taken not to use this regression equation 
-to make predict ionfe- for populations other 
than the one considered in this study.: - 

A sec?ond regression model fits 
KILTER'S run time to the problem specLfi- 
cations with the resulting coefficients: 

Y/KILTER = + ^x^i ^2^2 

where 

V/KILTER := run time for • KILTER Vn seconds 



x^, = number of nodes 
X2 = number of arcs 



and -the estimated coefficients are given 

jDy ■ - 

Bn = -17.63 with a standard devia- 
^ tion of 2-63 

B-, = 0.03267 with»a standard devi- 
ation of .00409 - . 

B2 = 0.01347 with a standard -devi- 
ation of .001376 

The estimated standard deviation of the 
residual for the above regression equation 
was 2.601 auid the coefficient of detemiLn- 
a1:ibn (R^) was computed to be .767. ' This 

" " " " - " 
The quantities of interest are actually: 

B, 



'0/a 



6, 



4.393 

.72 



= 6.1 



.01645 
0.0012 

0.00255 
0.00038 



= 13.7 



■ 6.-7.-^ 



Since these numbers are much larger than 
3 'and the Bj are approximate I'y normally 
distributed we conclude that Bq/ Bi and 
62 are all significantly greater than 



zero. 
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model iis not2as predictive as the pre-* 
viouS/«hut R remains relatively high;^ 
* here, 76.7% of '^fie variance can be ex-- 
plained by the number of nodes and arcs; 

When an independent variable repre- 
senting the cost range was added to this 
model the sample increased slightly 
to 77.7%. This insignificant -iraprpvetnent 
did not warrant inclusion of a third 
independent variable. HenceVthe simpler 
two variable model was selected* ^ 

,^ From thS^ two regression models,, we 
see • that I*PNEt seems to depend to 'a 
greater degree oxi the number of nodes than 
the number of arcs in the network 
(61 = .01645, $2 = .00255), whereas 
KIJLTER depends upoa the nximber of -nodes * 
to a much lesser degree (Si = .03267,^ 
^2 = .01-3470- Loosely spe^aking, this 
means that LPNET is more* node-dependent 
than KILTER, while KILTER is more arc- 
dependent than LPNET. * This result is ' 
intuitively ap]g>ealing since the primal- 
simplex algorithm LPNET works with a ' 
basis spanning tree (m-nodes) which must 
be maintained — thus the dependence on 
the number of .nodes. The out-of-kilter 
algorithm, on the .other hand, ^works pri- 
marily with arcs which are "out-of -kilter" 
— thus the greater dependency oA the 
number of arcs. 

Besides being used -for understand- ^ 
ing the interdependence of problem char- 
acteristics and program- performance, • 
.these regression 'analyses can be employed 
'for fbf^testing CPU times for problems - 
irl the population P^. This can be done 
as follows by making point estimates 
by substituting for x± and X2 vin the 
regres-sion equations or by setting up 
confidence intervals - for the . predicted - 
run times Y/LPNET and Y/KILTER. . 

For' example, a point estimate o£ 
the LPNET rxan^,tiine^ f or 'ja. problem with * _ 
456 nodes cind'll6C> arcs- (see problem 14^)' 
would be: 

Y/LPNET = 6.058 seconds 

The actual run time for problem 14 was 
6.000 seconds. Here. the error in the 
point estimate is approximately 1%. 

A 98% confidence interval for 
Y/LPNET would be: 



where Y, 



.98/2 °^ ^ .^tPNTT - ^I.niEr'*' *;98/2 



"LPNET the point, estimate run 
time/ S- is the estimated residual staii^ 
ard deviation of Y, and B 95/2 

■^In a similar analysis carried out- on 
larger* problems, the cost, coefficient ^ 
range became more important . . V " 



, standard normal deviate. 

. " Hence, in our case with 98% certainty 

6.058 - 2-303(.72) < ^lPott • * 2.303(.72), or 

Thus we predict with 98% certainty 
that for problems with 456 nodes ^d 1160 
arcs and any cost coefficients facing in" 
the range defined by our population LPNET., 
will take no longer . than 7.716 seconds 
and no less. than 4-400 seconds to find an 
optimal solution. 

Clearly, many other statistical-" 
vtests c?6uld be carried out, depending 
upon what the specific objectives of the 
expeirimehts were. , Since our aim ^ is to . 
introduce a methodology rathefr sthan ex- 
haustively compare KXLTER and LraET,^we 
did not ^conduct fu2:tHer \ testing J » 

5. Conclusions 

The statistical analysis' in Section - 
'4 shows that the code LPNET is clearly • 
superior to the code KILTER for problems 
within; the collection Pt« The conclusions 
are not uiiexpected since LPNET dominates 
KILTER. Nonetheless, we are able to mea- 
s\ire precisely the extent of. the super- 
ior ity and propose confidence limits with 
a statistical framework. Despite this 
analysis, there are many unanswered ques- 
tions about LPNET 's superiority to-* prob- 
lems outside of Pt but within collections 
Pc and P. It IS an open -giiest ion whether, 
similar concl^sdons would, be obtained if 
the objective of the comparative ^study 
was. to evaluate how well the cod^^reop- 
timize, that is , how well the cOM^re- " 
start from a nearby, basic- feasible solu- 
tion.^ Thus we caution the reade''r to treat 
these results as conclusive over set Pt# - 
but no t^ outside of this, domain. 

The primary purpose of this report 
was to develop an initial framwork for 
analyzing and comparing mathematical pro- 
gramming software. Obviously there is 
much 'unfinished work to be accomplished. 
A thorough study of problem generators - 
as relating to real-world examples is--. an 
important next step. If the complexity 
of real-world problems could be better 
understood, it might be possible, to design - 
problem ^gener ators , which more clo^sely. re— • 
^flept the characteristics of .realdstic 
' ^>5:o'b"leins * "The following idea which ' pi;o-- 
pofees^a synthesis of the two categories 
of test problems 'described in Section 2 
might be useful for building generators. , 

. Start 'by ch4>^ihg a representative 
t^'st case t from tne class of problems 
'under study, for instance, a particular 
nonlinear programming, design problem. 
Place perturbations on various parauneters,. 
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such that the cost coefficients, and ' 
call the resulting population Pf - (It 
is -left up the researcher >to select the 
appropriate, parameters and' ranges,) 
From this population a sample p is drawn 
and inferences, about Pt^are made.' Since, 

-the population Pt ,is defined to. bfe with- 
in an e-neighborhQpd 'o-f 'the original; ^ • 
test case% conclusipps^about^ Pt/ in some 
sense, reflect* a sensitivity analysis ' 
for problem t . . For "instaiice , the stand- 
ard d'eviation .:fun tiihe^ measures how 
sensitive the performance" of the code is - 
to small changes in the original data. 

* As -a secondary consideration^ it might 
be interesting to execute th^ programs 
from different starting. points . • ■ 

These concepts fall within the pur- 
view of experimental design. Since it 
^is important to minimize the computer 
costs for testings especially for large- 
scale examples, a soiond experimental de- 
sign .can reduce the variances of the 
estimates and thereby ^result in lower 
computational costs. As an example, the 

-simple ramdom sampling procedure which 
was performed in- this paper can'^^be re- 

.placeci by stratified S5unpling* 

Another important area for futiire 
research lies in developing a clearer 
view of the relationships between* the 
sets Pt* Pq «ind P. An estimate on the 
upper and lower bounds of CPU 'time for 
problems within Pt might lead to an ' 
".accurate estimate for average CPU time 
for problems within Pc- The same might 
be said for P as well- . 

The ultimate'' benefit of the sta-^ 
tistical frameitfork is for assessing ^ 
competing software codes or systems- so 
that .a systematic choice can be made as 
'to. the. best .technique for a particular 
user: Clearly this decision is^ a diff'^ 
icult. multi-attafibuted problem and Until 
these typ^s of decisions can be handled,., 
we must be content to list a profile of 
characteristics and perfoirmance measures 
for each code. From this profile, the 
user can select the appropriate tech- 
.nique for his or her particular needs. 

'Finally, we should mention that the 
working committee on ^algorithms fwCA) of 
the Mathematical Programming Society is 
actively engaged, in computational-lcrelated- 
research. (The authors are members of, 
the committee.*) The ambitious goals of 
the WCA are: ' 

1) collect a graded set of test 
, problems, 

2) a'Qt as a focal point for knowledge 
. * of computer programs that are 

1 available for the.same calculation, 

3) recommending "best buys" where 
several techniques are available . 

. . for the same calculations. 



4) encouraging persons who" distribute 
programs to meet certain standards 
of portability, testing, ease of. * 
use and documentation^ and 

5) define guidelines for conducting 
and reporting computational exper- 
iences . 

One of the most important, and mo^t dif- 
.ficulti goals is the last " to. define 
journalistic guidelines.^ The imposition 
.of fair guidelines could markedly improve 
the state of computational work by pro- 
viding better information to researchers 
and software users. 
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^* , * '" Abstract ' 

We discuss different a^^proaches to evaluating 
opti mz ati on routines and- describe a particular \ 
nethod /which i^^es paraneterized test, presents.. Vfe 
illustrate this approach thrpu^ a siiiple case 
study of three well kncMn tano ot i strd ined optiiniza- 
tion routines applied to three parameterized test 
probleras. In particular we display our results as. 
a set of gra^iis. 

1. 

■ ' . ^ ■ ^ 

Evaluating mathematical routines* is a diffi- 
cult task, and cne that requires both qualitative 
aiad quantitative measures of performance. A fun* 
damental requirement is that the testing environ- 
ment sijnilate an actual envi r onm en t of use since, 
•if it did not, the evzduation wjuld be valid, but. 
in all likelihood, irrelevant* Furthermoore, the 
overall quality of a code- can only be gauged after 
investigating a broad range of Issues, for exanple, 
efficiency, rpbustness, tasability, 'usefulness of 
doctmentation, ability'to. fail, gracefully in the 
• presence, of user abuse, rounding error difficul- 
ties or violation of imderlyirig assuitpticnsp A 
testing med^ iisually conoentrates on efficiency 
and rcixistness, evaluating these by exercising 
^the code cn a set bf tmbU chosen and hcpefully ; 

.j^ealistic problems* . .. . . 

-v.- .,• i ..... . ; ; ■ > ' >: •■ ■:■ ■ ' ■ 

' ^ / Oir aim in: this paper-?is to discuss some of " 
the issues pertinent to the evaluation of cpti- 
mizaticn routines, and to describe an a|?groach 
vinch etploys parameterized test problems, first 
ii^troduced .for the purpose of evaluating routines 
for numerical quadrature by Lyness and Kagancve 
[1] „ Vfe describe in Sectioais 2 and 3 our par^ 
ticular motivaticn for and usage of such functions. 
H> illustrate this approach three well known, cpti- 
. m a-za t io n rou ti nps vaere evaluated in a-^siitple case 
stucty. Section 4. and 5.^ describe the routines 
and- test problems,, and present the resul-te in 
^ra^iiical form. We believe that this cajsS^js^udy 
gives sacae iiiteresting exanples;of how' linoan- - - • 
. str7^iTTf>f^vOptiinizaticgi Routines bj^iave vfoen applied- 
" to pai^ameteri^^ test prdbiems. ae study is, \ 
however, very limited in' scope, arid we discuss , 
sdne of tiieSe Ximitcrticins in the concluding. ' * 
••; secticn. • " ' ./ 

^Wadc gerf o iiued under the auspices of >U.S, Energy 
Researcdi and Development -Mministration. , 



2. DIFFERENT APPROACHES TO TESTING 

Tt> date the most ca m mi method of evaluating 
. cptiraization routines has beccme known as ^hattery* 
or 'simulation' testing.^ The first really occtpre=^^ 
hensive study along these lines is that of 
Hill s Lr ail [2]. Battery testing has two basic coro^ 
ponents, namely, a set, of, test problems given by 
an objective functic^ and a starting vector/ and a 
. number of measures of performance. ■ ' . 

Test functions are chosen fecm^the literature, 
or from real life applications, usually because 
they have sane prcminent feature,- such as a curving 
valley poss'ibly helical, a singular Hessian at the 
rainimura, badly scaled variables or- large dimension- 
ality, Tbe choice of st2urting points is, of 

.course, crucial to perfontance. Many routines per^ 
form well from the standard starting points, How- 
^ever, ^as discussed . in [2], the use of a number of 

, widely dispersed starting points, reveals nuch 
aborut the strengths and weaknesses of. a code, for 

Jexartple, how robust it is, the suitability of it3 

* convergence ^fcriteria, or the code's ability .to 
handle long^ searches tbaxnagh nan-c[uadratic regions. 



A variety of different measures of 
have been used, "ffie mast cciimai measure is the. * 

, number of calls, tcr. the. routine .which develops, in^ 

- JEannation atout the* functich, .or • the numbiar oif 
equivalent function evaluations;' since the cost oif 

' gradient information is hardly ever: the eg^valent 
of n function evalxiations (y^iere n is the problem 
ddmensim) , the c^:propriate weig^iting of gradients 

•leads to the DCa or Homer unit scheme of 
Hillstran [2] . Other measures include* overhead 
and average rate bf convergence. ^ 

Whilst battery testing yields very valuable 
information about the behaviour of optimization 
routineis, it is nevertheless subject to limitai- 
. tions viiich often make a clear ranking of methods 
^difficult to discern. *^irst> it is difficult to 
'knc^ hew modi confidence should be attached to the 
measures of performance, since slight variation .of 
stUartiiKf point' or geometry. of test problem" can 
lead to substantial variation in a 5^:±6rmance , 
measure^ Second^ when starting-points, are .varied * . 
substantially >. one can, in effect, get very differ^ 
ent test problems. For exaziple, Bosenbcocik's 
function f(x) = 100 (xf-X2) 2 + (l-rxi)2 is - \ 

more difficult to solve if started at the point 
(-2,2) than if started at (2,2); in the former 
case, a routine mist follaw a ste^ curving 
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vsLlley, particularly taxing at the point (0,0). 
. Thus, vAien evaliiating a rtxitine's ability to cope 
with a» particular feature it nay not be advisable • 
•[to average a performance ireasure over widely diff er- 
ring starting points; however, if no averaging is 
done/ one may bje swan^^ed with nunters, 

.In attenpting to deal with sone'of these dif- 
ficulties/ we have utilized parameterized test 
problems, introdxxred originally into the evaluation 

' of routines for numericeG. cpiyirature by Lyness and. 
Kaganove [1]* Vfe will not, however, use the 
Xyness-Kaganove term perf armaunce profile testing 
■feiiich involves a sonEwhat specific xisage of 
paratteterized test problanis to ranX "black-booc" 
rnfttheroatical routines. Oliough we also want to cctn- 
pare routines, our usage of parameterized test 
problenB is oriented tcward the gathering of infcr- 
raation cn the pe rf ormance of an algorithm, '^idth a 
view to further develqpnnent of its icplenientation, 
Thus our bias is toward the develcjpnent of tools 
to aid the process of algori-dun developoaent rather, 
than tools to eud nathenatical sbflS^are evalxaation; 
thou^ we are also ocncemed with the latter it is, ' 
in our opinion, a much harder problem. In tKe 

,next sectijon we discuss our method,- 

3. OORAPEaiACH 

Test prdbleros are chosen in vtoch a proninent 
feature can be varied* by changing the value of a. 
parameter X occurring in the prcblems mathennatiCal 
fcrnulation. We have, departed frcm the Lyness- 
K2kganove approach which is to parameterize a prx±>- 
lem'in such a -way as to provide" many different 
•incarnations'^ of a test problem, all of a^sproKi- 
mately the same level of difficulty, by parameteriz- 
ing e.g. the position of the peak of a unimodal 
function butwpot its shape. Varying cur parameter 
alters significantly the overall difficulty 
of the*prc4>lOT, Uius one can test a routine's 
effectiveness with re^sect to a particular fea- 
ture and "study i±e routine's perfor m a n ce as this 
feature beccmes more and moore prociinent. Par 

■ example, the. feature may be ai steep curving valley, 
whose Steepness or curvature increases with X. ' At. 
the same time measures of' pisTfarropOice for values ' 

' of'XjLn the neigflibarhood of any particular, .value, , 
say Xv will give an idiea of .the va±iRbi1ity- of the.. 

" perfbcmance measures * ' 

Starting points are, of c o urs e, critical to ' 
the TniniTn'^r^->'T<^ process and often bias the path 
of searc±i. In addiiiion to running each test using ■ 
a cQ tiy entia n al or fixed starting point, ^the 
initial points were .varied using a randan iioriter 
generator. motivation, for this. approacH is 

twofaLd: .^ 1) it was desired to obtain the "spread" . 
between tie 'roaximiim and minimum number .of rs=r,ns to . 
PCN and 'thus det^miine how sensitive each routine 
is to a change in the starting point; and "2) by 
obtaining average vaiues and- using these for ccm- 
.-,parison purposes, it was hoped that the results ^ 
would be less biased with respect to the starting - , 
points. ', ■ ■ ' - 

Clearly, ^an iitpartant consideratidn is hew " 
. the; points should be varied. We chose to vary - 
starting points' within a "T^dx" ; surrounding the 
ccmentional stiarting point thus p ro d ucing' dif- 
ferent initial points but the same genercLL 



starting location with respect to tl^tqpogr5^)hy. 
A hypercube of cEmension 0.1 units was used for all 
jthe tests in which the starting points were varied. 
Another iii^xartant aspect: is , that the same starting 
points were used for each run (i.e., for each 
V2due of x*) by resetting the randan number 
generator. 



Olie statistics gathered for each test function 
include the following: * , 

1. A table for each optimization routine us- 
ing conventional starting points. The 
number of IfCN calls, the number of itera- 
tions, the solution point, 'gradient, and 
functicn value for each X are^given. ^ 

2. A table for each optimization routine los- 
ing the random 'starting, points ,': Ohe 
average, maximum and minimum nurnber of FCN 
calls, and the average number of itera- 
tions for each X are given. 

3. Plots of the maximom, minimum, and average 
number of FCN calls versus X for each . 
routine used. 

4. A superin^posed graph of tHree j>lots (cor- 
responding to the three routines) of 
average number of FCN calls versus X • - 

alie graphs v^e plotted using Fortran^ subroutines, f 



A CASE STuny 



In order to illustrate these ideas, a siir^sle 
case study was carried cut involving three routines 
and three parameterized test problems. 

4.1 Routines Used , \ / " 

' . • ■ ■ . -. 

Olie following three unconstrained nonlinear 
optimizatiort routines were used in this stucty. 



OGMTN - a. m od u larized version of, the 
function minimization Harwell Library 
routine "\^8A written fay R, Fletcher 
"■ [3] \diich uses the Fletcher-Reeves' 
■ ■ -version of the. conjugate gradient . 
"technique (see [7J) ., 1 • ' *• \ - 



__ 2. BPGS - a modularized version of the- v 

Davidon-Fletcher^Powall quasi-Newtcn 
function minimization algorithn [4,5] 
wit]? a BEGS (Broyden-Fletcher- 
. Goldfarb-Shanno) [6J update to the 
Hessian (see [7]). 

3.- OOOPTR - a modularized inplenientation of 
" . C. Davidon's [8] optimally conditioned 
algorithm for calculating the miniirum 
* of a'function of , several variables, 

A user-si^jplied subroutine PCN vMjch evaluates the 
function valxae F and the cccponents of the 
gradient G at. the point X is needed by each of the . 
'above 'routines. , / ' ' \ ' " - ^' / . ' • •• . ; 

. *ln addition^ the user most siggly the follcitf- 
jqg* information in the calling program to OCMEN: ' 
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1. the xunber of variables (i.e. the dimen- 
sion of X) ^ 

' . ■■ ' »' " 

. 2.* the initial estimte of the solution vec- 
tor X. / \ 

3. . an estimate REK of tlje eaqoected reduction 
in I? (used on the first iteration only)^. 

4..^ the accuracy required in the solxiticn, ew 
^rSD^ first condition for a normal return 
is that the differences* between the coo-, 
ponents of twD successive estimates of \ 
•\ the solution are less than ct 

■ ■ • 
- 5. an rifViitinna l accuraQr reqilireraent^ • 

Ihe second condition for a nornal return • 
is that >the gradient norm is less than 62 

6. ^the limit cn the nunber of r^^iig to the 
function evaluatipn subroutine FCN^ If, 
the limit is exCeetJed before a minimum is 
. attained/ CGMDY will tenninate abnonnally 
with an appropriate error, code. ^ 

Fbr all of the runs>"^Rra was set" equal to 1.0; 
and C2 were . set equal to 10"^. Pte all three of 
the routines, the limit cn the nunber of calls 
"was set at 200, 500, or 1000 depending cn the test 
function. - 

Olie rqutine BFX3S requires the. following inf orc- 
mation ^ from the calling pQDogram: 




/ 



1. & 2 

3 



(same as for OGMIN] « 



4. 
5. 



the accuracy required in the solution, g. 
A rtrmrpJ. return occ u rs if the sum of the 
absolutei^aOSaes of ithe caipcnents. of the 
solution difference andfdirecticin viectors 
are'both not greater than e. * ^" . 

an estimate of the minimum value of 'F{X) . 

the limit on the nuirber of r-?^^^^ to PCN. 



Hi all cases, e was set to 10^^ and the estimate of 
♦the minimun. vjas set ecgpial to 0. 

' y I , ■ * ;■■ 
rIhe necessary/parameters iri -H-tf^ '^r^l 1 1 Tvj ' prrw 
"~ Jbfae cptimizaticn routinec OOgro aa^e:. ; 

• ' 1. s 2. (same.. as for C33irN)-.* ■ • ^ ' " 

3. the limit on the nuniser of POJ calls. 

■ 4. the accuracY sou^ c. A ixarmal return 

occurs if g%g < e, vtere H is the current " 
^JFCoodmation to the inverse hessian, and 
g is the gradifint at the current iterate. 

5, an estimate on the lower bour^ of the 
,&nction. e was set to 10"^, and the 
lower bound estimate was ^t equal to. 0. 

4.2 ProhLsms Selected • c 

■ ■ -The following test functions i.were parameterized*-' 
and^implenEnted in the evaluatiorio^ the above al--* 
gcrithros (Xq represents the f^j^ yM t r yiaT or originaLL 
value of the parameter) : 




tosenbrock test function 

• " cV, oo nv a i l irtrvi l starting paint is (-1.2,1.0) 

Hje func±ion F has a global miBiiiuti at 
' (1.0/1.0). me randan startiing points were 

generated over the ccnpcintot intervals 
.Xj^(-1,3A-.1}V X2(0.9,l.l). 

2. PoweGLL Pararoetjerized badly scaled function of 
two variables {X^ = 10^,000) 



a. 


^1- 


b.- 


F ^ j 




3F 


c. 






3F 




" 



d. ccnventipn28l starting- point is ,(0.0,1.0) . 

Rar X = lO**, the function F has glcbal minima 
at (I,098xl0"5,9^i06) and (9;l06,1.098xl0"5) . 

tlhe rantton startiag'^poiiits were generated over 
2 carponent intervals X, (-0.1,0.1) , 
(0,9,1.1). ^ , V. 



^ Perturbed Quadratic IXmction ' 
a, F = + 2X^ + 3:^ + 4:^ 

+>3X^ + 43^ + 6X^) 



3F 

ax. 



\ 



c, "fixed. starting point is (10,20,30,40) 

^Ihe function F has a gloJbELL ininiiium at 
(0,1), 0,0)^. Olie randco starting points were 
generated over the carponent intervals 
Xi (9.9,10.1), X2(19. 9,20.1), X^ (29.9,30.1) , 
X4 (39.9,40.1). ^ . 

AU-tbe tests ^ resorted irt this^p^iS: were run 
-in doable precision X'rXS significant digits) on an 
IBM 370/195: ' The numericatl .tnderflow ani overflow, 
and the divide check intem^Jtis were si^jpressed. 
!Ilhe random starting points were obtained iising the 

iipplied "Mathematics Division library random . 
nurcber generator G552S. ■ * . * 



5. TEST RESULTS 

Graphical results desc^ribed an Section 3 cire 
given in i^ppex^dix 1 for tri^ ^above problems. (Tab- 
ular result? iare not given^^l^^caiise of space 
lindtations-) Oily the initiaf'portions of the 
.graphs are reproduced here, these being sufficient 
to illustrate the points raise<i during the dis- 
: cussiarviin> Section 2.. ' ' ^ 

a) Parametrized Rosenbrock»'s function 
F(X) = U-Xi) 2 + \ (X2-xt) ^ ' 



For X - 100 this is' a well loiown, difficult 
to miniinize test function^ v^ch has a descending 
paraboliti; valley as its dominant feature- TSie 
starting point (-1.2,1.0) is chosen to bias the 
■ search dowi the vall^. -.The parameter A was varied 
from 20 to 1000. in steps of 20; increasing A in- 
creased the difficulty of the parcblem by ma3dLng the 
valley steeper * The limit on nuirber of calls to 
FCN was set to 200. 

Figure lA shows quite clearly that the perf or- 
itance of CGMIN and BPGS are ccnparable, with OOOPTR 
performing substantially better. Figures IB, IC ' 
and ID si^ the sensitivity to starting points of 
each routine. In Figure IC note in particular how 
substantial^ the variation can be for certain values 
of y, e.g.. for X - 120 - Such a jshoice of A in a 
battery tkst could suggest a very misleading rank- 
ing of the methods- Althou^ we have riot atteitpted 
to track down the. reasons for the poor performance * 
of the BPGS metl^ in certain instances, such detec- 
tive would undoubtedly lead to iiiprovements in 
the impleniaitation'. 



b) 



Powell's Baidly Scaled Rmction^ 

F(X) = C^XiX2-l)^ + (e"^! + e"^2--. 1,0001)2 



the starting points were varied, the spread be- ' 
tween the maximum and mininum numnber of calls was 
very small in all cases, indicating that the 
initial point is not extremely critical in this 

C£t5e. 

6. CX»)CLaSIONS 

CKsr primary aim has been to develop a testing 
method and software tools centiered around - ' " 
parameterized test problems and graphical display 
of results, these being designed to aid the process 
of algorithm develppn:)ent- We have illustrated this 
method and shewn the use of the tools in a siir^jle 
case study. :The results of thiis study provide 
interesting exarrples of hw qptiinization routines 
behave v^ien applied to parameterized test prcblans. 
The case study, of course, suffers frctn a number 
of drav«tocks v^*iich must be remedied ' a practical 
evcG.uation of optimizatian routines- For exanple: 

a) Each routine studied enplcyed a different oon^ 
vergence criterion* T!his intxoduces a lack 
of uniformity in the ocnparison. 

b) A more detailed investigation should stuSy 
each routine's performance initially, at inter- 
mediate stages, and on the final stage 

it is near the minimum. 

r 

cX ^ A broad set of t;est problems should be xised, 
in particular problons with large or variable 
dimension. 

<£) For each test function more than one' starting 
box should be used in order to sanple a wider . 
region of the topDgraphy. 



The* test problem is a tiough sha^jed function 
for which A« was varied from 10 to 1000 in steps of 
10. Changing \ makes this function' more badly ^ 
scaled and thus more difficult.. to minimize. The - 
limit on the nunber of r^lTs was set to 1000. 

Figure 2A shows that a ranking based i^son one 
.particular value of . A ,can be misleading- For 
X < 180 the performance of OCOPTR. and COmj are 
conparable, with OOOPTR sonewhat superior. BFGS ' 
perf o rms . d ec id e dly worse. Hcwever as X increases 
tlie-perforronce of OGMIN^ deteriorates rapidly. 
Furi±ennore, Figs. ^2B and 2D dpnmstrate that for 
CC^HK and OOOPTR results aire sensitive to starting 
points viiilst Fig. 20 shews that the performance 
of BEXS is relatively insensitive to starting 
points.-. 

c) Perturbed Quadratic Function 

^. F(X) ,~ Zi)^ + X(X^-h 3X2'+ 4X3 + 6xl|) > 

This test function is a parameterized ccmfai- 
nation" of a quacbratic and a quairtic function. 
Wien X — 0, tiie function is a sijiple quadratic and 
is ieasy to minimize. However, as X increases, the 
quartic becomes more and more significant- X was 
varied from 0 to 1 in st^ of .025. 

Except for the case<.x - 0, OMCN proved su- 
perior to BFGS for this problem- Agedn OOOPTR 
performed tiie.best and for X > 0 wels quite insen- 
sitive to the particular value of X. Also, when 
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r ■ - ; Legend -for Figiires ■ • • . ^ ' " - 

a) Figures lA, 2A, -3A: Average number of calls taken over set of startijig 
points, plotted against \y Superiitposed .plots, are given for 03SIN(*) , 
EFGS(§) .and OCOPTR (#)-. . ' , : . v ; / 

-b) I?anaining Figures: Bar each routine, mxinum W , Average (0) and 
Minimm (X) number of calls for set of starting-points, plotted 
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Provided herein is a sumnary of the comments made during this panel discussion*. 
Although every attempt was inade while sutirorizing to portray accurately the 
essence of each contribution, inaccuracies will persist: Responsibility .for 
all such inaccuraci^ or misrepresentations lies with R, Jackson, who prepared- 
this summary and functioned as chairman during the discussion. Names and 
addresses .of all contributors to the discus^sion appear at the end of this 
■text. ■ .■ - - ■■ . ■ ■ • 



R, JACKSON: I would like to welcome you to the 
panel discussion on "Issues in the Evaluation of 
Mathematical Programning Software" sponsored by 
the Coninittee on Algorithms, (formerly the: Working 
Cormrittee on Algorithms) of the Mathematical 
Prograimring Society. The Committee on Mgorithms 
was created by the MP Society in 1974 with the 
charge to. concern itself witfr knowledge, informa- 
tion> coramini cat ions, recommendations, and other 
actions on mathematical programming alsorithms , 
and testing methodologies. The ambitious goals 
of the Committee were identified to be: 

. 0 ensuring' that there is available to the ' 
MP community a suitable basis for comparing 
. , algorithms (e.g.', a graded set of test 
, problems); 

0 acting as a focal point for knowledge - 
of- computer programs available for . 
general computations; ; ^ 

\ o reconnendi'ng "best buys" where' sevieral codes , 
are available for the same computation, and; 

0 encouraging developers to^ meet certain 
standards of portability, ease of use, . 



and documentation (e.g., by producing , 
"standards" or guidelines for the repor-. 
' ting. of computational results). _ 

This session, then, is. being viewed by the 
Committee as another opportuni ty . to create a . 
dialogue between the. committiee and the math- 

.ematical progrannringxommunity.' We hop > to 
focus the discussion today around the question 
of guicielines for the publication of the results 

.of computational experiments, an item that I just 
mentioned as being one of our goals. With thaiJi* 
then, t would like to throw the floor open for 
questions either from the audience of; from the 
panelists. ' 

R. DEHBO : I have a comment to make on the paper 
that Dick O'Neill presented in the morning session 
In that paper, he conjectured that ^randomly, gen- 
erated .problems tend to be more difficult to 
solve than real-world pmoblems. It has beien 
my experience that the i:everse.of that conjec- 
ture is true for Knapsack problems. . I wondered' . 
if anyone else had a similar experience. 

Rw O'NEILL : I'll mention quickly that I can 
generate problems that will be solved in one 



progranming algorithm. So you can generate 
problems that are very easy to solve. I didn't 
want to make that conjecturer.as strong as it • 
might have -^eem^, and I .am sure that there are 
classes or sub-ciasses. of problems that have ^beeri 
excluded, but isome of the work we have done at - 
.LSU indicates that there is some truth to this 
conjecture. In addition, Darwin Klingman and 
Gordon- Bradley also agree therfe is some- truth 
to this conjecture. .-One point that can be made 
however, is, that if you^xian develop a problem 
generator that generates difficult problems, you 
are in a good position to test codes that will 
eventually be run on real-world problems,.,^. 

C. WITZ6ALL : In Jim Ho's lecture today on stair- 
. case problems, he mentioned . that Martin Beale found- 
-staircase problems td be rather more difficult to. 
- solve^than -general, more distributed, linear, pro- ' 
. grammfng problems. This could perhaps be a piece 
of evidence that works the other way, because- ^ 
staircase problems a ret highly non-random. 

4 

H. 6REENBERG : But the proper comparison would be 
to have a generator generate staircase problems,' 
so you would still be comparing staircase problems 
• with staircase" problems. It's just that one has 
random entries and the other has some real data, " 
You would not compare a general LP with a stair- 
case problem. 

R-JACKSON: Is there a generally agreed upon 
detinrtion'- of what a randomly generated problem 

. H, GREENBERG : T think it was at least alluded to 
in the presentations of O'Neill and Mulv.ey this 
morning^ where they referred to sffecific structures 
with controllable parameters. But the situation 
is that you have a set of controllable parameters 
like sparsity and row counts and that the- randomness 
appears in costs and right-hand-sides.. • 

J. FILLIBEN : I would like to make a coirnient on the . 
relationship between the typical math programming 
problem discussed earlier in the afternoon and a 
^measjurement process that one usually runs into in 
scientific experimentation. What you are really 
doing when you discuss what should and should not 
be controlled in a mathematical programming problem 
generator is specifying a" measurement pfrocess, A . 
well -defined measurement process .has all of the \ 
components of that process identified; for example., 
the donrain of variation. \It should be noted that ' 
this is different from the simple injection of - 
randomness into a procedure. For example, in the 
Simplest situation, one lets everything vary with 
nothing under control, and from a statistical point 
of view, you are assuming least, and you end up : 
with least.;- A better situati on is. what statisti- 
cians refer to as a "constrained randomization 
problem", where'certain of the problem components 
are removed from the random domain and put. into the 
deterministic domain: The remaining components 
that we can' f control we then randomize.* In es-. 
sence, you have recognized that there are certain - 
problem parameters that .are of special interest* 
and in this case these are the ones that are to 
be contrqlled. What we can' t control , we'randomize. 
Two of the three talks that I h'eard this afternoon ' 
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constrained randomization. The other deal th with 
a completely random -situation: But the point that 
I really want to make is that rthere is a 'relation- 
ship between' testing by setting up problems in « 
order to check out algorithms, with tbe/corres- ' 
ponding problem in the scientific community of 
• defining a measurement process' by specifying 
exactly ^hat -conditions, .should be placed on the 
variables that we intend to control. It has to ' 
do with much more than just having a random n urn- . 
ber generator. 

C. MYLANDER : It seems to me that in this discus- 
sion of usTng randomly generated or real -world 
test- pr6blems, we haven't\made it as clear as we 
should just what the purposes of this testing are. 
,.-It seems to me that we aral toting codes and ' ■ 
algorithm for two purpos^rT JDne'^is' to have a 
code certified and saf^^r the us^r to use and 
the-second is' to compare \al go rithm^ for solution 
strategies ,;S0 that developers of codes can decide' 
what is the best solution strategy to builci into 
a code. I would, like to hear that kind of issue 
discussed a little bit more. From the point of • 
view of a user, I'm willing to pay for computer - 
inefficiencies so that Z can use a small, collec- 
tion of codes rather thafi a- larger collection of 
specialized codes. ' And then, on another topic,' 
the one abon't efficiency of a code on a particular 
kind of test problem, I think my experience is 
that the way a problem is formulated causes ^uch 
greater variance in computer efficiency than 
either the codes or algorithms themselves. 
«. 

R. DEMBO : I think the two different objectives 
you mentioned, that of designing safe codes arc 
designing robust , codes, conflict to some extent. 
If you -want to conduct isxperiments ip order to ^ 
have better equipment for designing cod^, you 
should probably use the ideas, presented ffe^ Larry 

^ Nazareth?s talk, this rooming^ oecause there-you 
are testing specific properties' of an algorithm. 
On the^other hand, if you just take arbitrate 
problems and generate them, you have ;io. idea 
whether you are testing- to. see if there is a 
very steep valley you are- going into or what 
happens if you elongate this valley, . So, if you , 
want to design algorithms,- I would say what you 
should do. is pick problems- by hand and perturb 

- them.- If you want to test robustness however, 
you would' probably take as wide a class of ^ 
problem as you can genierate. - ButVthat means, 
of course, that if you are really interested in 
testing robustness, that's probaffly ^11 the 
information you get. . ' • . 

H. "GREENBERG : I think there are at least three 
.purposes for doing any kind of experimentation, 
along the line's we have beisn discussing. The 
first purpose has to do with program correct- 
ness:, to see whether' the program is correct. 
I really, don't know how to do that, especially 
when you consider that the se4: of inputs that 
absolutely .would guarantee correctness is larger 
than the total number* of bit patterns in the'input 
data. A second* reason is. to do benchmarking for-: 
the purpose .of selecting an algorithm' that you are 
.going to buy to sol ve problems.. I think the £• 
perturbation method that Larry Nazareth .proposed 
is a good- approach to that problem. A third 
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cover tne poorer aspects or a given ajgontnm so 
that that poor aspect. can be fixed, thereby yielding 
a better algorithm.. I should note. that you" want t o ' 
be careful not to gear algon'thm. improvement towards 
a worn-out. test pVoblem,--! tke Rosenbrocl^"'s function* 
I. thinjc too many algorithms have been desigrled to 
do well on. Rosenb nock's functton, especially Ro§en- 
brock's algorithm; but it is not clear to me that 
using Rosenbrock's function Will necessarily do 
aoything for improving your ability- to st)]ve. non- 
linear programn^'ng p^blems in general. So I 
think that we have to take great care in designing 
a collection of problems that in some way contri- 
butes insights into algorithm improvement. 

C. MYLANDER: r think^ we have to check the battery 
approach ^against sporadic selection" of all sequences 
of test problems that are available' to code devel- 
opers. - . ■ ^ ■ 

H^. ^REENBERG r >'.Bdt -in that case you only hjave a 
doren hand selected problems wi th 'no indication 
•of their properties or what happen^ when you start 
to vary things. I-think the generation appVpach 
is more helpful in that avenues^for research in- 
clude getting a handle on a reasonably small set 
of -parameters that is large ^enough to capture every - 
aspect of problem structure that in some";sense 
gives you reasonable representation of the real- 
world'^class being ^addressed. ' , . 

D. SHIER : I wanted tc5 follow along witb the ^ 
earlier topic of defining "randomly generated" 
problems ^ sometiines feel a little queasy about ' ' 

,the defijiition of what is a randomly generated 
'problem and what one actually gets'* put* I'd like 
0 give you an example. Suppose .1 want to have 
five randbflily chosen numbers that sum to one. 
One way to'do thjs is to generate four of the 
numbers that sum to l3ss than one and then sub- 
tract that sum^ ¥rom one to get the fifth number. 
^Another way to do it is to generate five numbers ; 
randomly; and , then divide them by the sum. How- 
ever* that will hot give you a uniform distribu- 
tfon over the^ space of. ^interest. My point is that * 
I think we have to look very carefully at- what 
exactly constitutes.^ random problem. I also . ' 
•think that whe'n you start, to specify side condi- 
tions, especially the KuhnrTucker conditions for .. . 
a" given problem, '^there may be some problems there 
that I aon't believe have been^addressed by those 
who have produced test problem generators. 

fif. O'NEILL : .1 agree with- you^ In one of the^ 
experi ments that I conducted, the only* thijig that 
changed in a problem from one run to another was . 
that the cost row was generated by way of the 
Kuhn-Tucker conditions, as opposed to being raa- 
domly .generated with values between 1 and'lQO. 
In one comparisonj the Kuhn-Tucker generated problem 
took significantly more time*- but in qnother case> 
the times were about the same. So I agree with you 
that we should try to determine the properties that ' 
these^-tes-t>problem generators have;- but that's a. 
difficult area. . . . 

J.>FILLIBEN : I might mention thAs poirTt: that^ 
there are a wide.vas^'iety of -statistical tests , 
that, one :can use for checking varidus random 
numbter generators. . ' v 



Muivey generatea tne tesx prooiems^Tor rne paper 
we presented this morning, we discovered that ^ , 
another pfoblem with generation comes into .the ^ . 
type of sampling. that you want to do as well. We. 
wanted to do simple random sampling, so we had, 
in some^sense, defined our- population. We there- 
fore picked a problem at random from that popula- 
tion. Now*, y/e h^d each random variable varying 
uniformly accordihg;to certain distributions but 
rm still not sure if what we did can properly " 
be called simple random sampling. I wish I were 
sure of that. ^ * - ' 

R. OTTEILL : ' It's not clear whether test problem 
generators have any sort of inherent biases in 
them and I'm. not sure whether you can ever deter- 
mine, that conclusiv&ly. 

R. DEWBO : ■ They 'do Jtave ^'bias: feasibility. 

'H. GREENBERG : There are other kinds of biases 
' too. Tt\ere are generators that may be biased, 
toward certain kinds, of algorithms, so they would ' 
necessarily -show that kind of algorithm in a 
favorable light* That's very little^ understood 
and I\; think' one of the interesting avenues 'of 
research is d4rected toward understanding genef*- 
ator biases. - . ' 

' Ji Wt^EY :- We have to- be realistic. about some of 
these things. Certainly- It's very important to' 

^analyze the differences between real -world prob- 
lems and generated problems'but I think that as. 
researchers if we're^ever going to solve large ' 
numbers of problems, we'll have to use test prob- 
lem generators. I think that foV the: future we 
can draw some parallels from some of the other. . 
scientific disciplines. ' For example^ there is 
an American Society for Testjng Materials that 
meets in this building, in fact, which is res- 

sponsible for developing standards, for'^esting- ' 
materials. I believe that is. something that 
will eventually happen to software: it will be- • 
come 'an .engineering fSnctton to test prtjgrains. 
But I think until such, a society oc institute 

o's created, we're-going to have to ,look at these 
portable generators for methods of compajing ■ y 
our algorithms. They're much easier touse^ 
than trying^to shift large amounts of problems 
back and forth between researchers. I just 
don*tothink that's' realistic at this time. ^ 

R. DEMBO : But that doesn't detract from testing 
the statistical properties of test problem gen- 
erators. . - 

■■.•■>••* ■ - . ■ . 

J. MULVEY :. No^ in fact we should look more at 
the- statistical properties of generators, be- . 
cause I think that's the way the research is 
going to go. \ . , 

. ^ - ■ ■ : ■ ■ . ■ 

D. SHIER :^ Let me add a comnent to tljat. At one 
time we had a summer student^ho was doing some * 
work on scheduling problems and had occas'ion -to use 
one of the standard pseudo-random number generators 
Turned out that whenever the number -of>4 terns being 
scheduled was; divisible by 2, very unpredictable 
results occurred. For example, regardless of the 
number of items being scheduled; in some cases 
he had hundreds of classes, it all could be 
scheduled within two hours. It turned out that' 



that it really is necessary to dig down and loolc. 
at the random number generators that we're using. 

J. f'iLlIBEN : This brings up the point. that a given 
rasdom number generator is only "random enough" by 
coinparison to. the purpose for which it is to be. 
^ used; and it sound like,. for mathematical program- 
' "niing-test problem'^generalfioni'' there are some strong : 
'dependencies on veiy subtle properties of* random ^ 

• number, generators that are being used> Furthermore, 
these properties may in fact depend on the algorithm 
on. which the problem is^to be run. So that a given 
random number generator may be alright for a certain 

. type of. algorithm but not alright for another type 
of algorithm in the worst. possible case. So getting 

• back to my. original point, T.d like to emphasize 
that, whetfier a:given set is random enough really 
dependS'On for what purpose that set is to be used. 
This.'in turn implies a' need for a thorough under- ^ 
standing' of the variot^s kinds of mathematical pro- 
gramming algorithms and the properties each al go-. 

• rithm as they relate to test problem-" generation. I 
.don't think that such understanding exists yet. 

R. JACKSON : ' This appears to indicate, then, that in 
addition to investigating a.lgorithm performance on - 
particular classes of problems, we should ajso in- 

. vestigate the correlation between a particular algo- 
rithm and the particular random number generator 
used in the test problem g'enerator that produced 
the problems; on which an algorithm will be evalu- 

. ated. And if this is true- then we have identified 
another a^/enue for research in evaluation metho- ' 
dology- ^ . • ' 

. . - ■ ■ f . . . , ■ 

R. DEMBO : I would like to sijggest that we. move on 
to an "important 'topic with immediate consequences; 
the is^ue of guidelines for pub'lications. I think * 
it's very important that we tighten up J^e criteria 
usejd for selecting articles for publication when, 
they contain computational results. . I would like 
to put .it out as' an "open question for the. audience 
to inform us if they have any suggestions as to • 
what guidelines .could be implemented immediately. 
Certain guidelines are* to- my. mind; oBvi-bus. ^ For 
example,,.. specifying a compiler when' yop tun a prob-.> 
lem; specifyiTjg convergence criteria wheh^dealing 
with nonlinear programs;- but l icould gcTorTand on. 

.Rr, JACKSON : This general isSuejpf guidelines is 
a very difficult one and I agree- with Harvey, that 
trying .to discuss the general question of publica- 
tion guidelines might be difficult, Howevisr, ■ 
certainly there are aspects of that question that - 
we could get into. 



MULVEY : I think we could go right to the ques- 
tion of test 'problems. For example, should we 
require researchers to provide test problems to 
other researchers when they publish results? 
think that is^ question that we can pose for re- 
producibility, >^ ^ ' 

R. JACKSON : Before we move into the discussion of 
the collection of test problems, I would like to 
provide a little background. In the first place, 
we are assuming that in. the absence of any other. \ 
well accepted method of comparing one algorithm 
against another, the test battery method is a 
valid one. Furthermore, given the fact that the 



tnere ougrn; xo oe procucea a well accepted set or 
perhaps even a graded set of test problems that 
are to be used by all researchers. "So the* question 
■ becomes , then, how can such a set of test '.problems 
be created? This has been looked inter by a number 
of groups over the years; SHARE dfd that, Wolfe v, 
did that, along with many 'fOthers. The question^ '''' 
confronting the committee now is. what kinds ~of 
way^ are available to produce such a graded set 
"-. of test problems? One suggestion is to require 
that anyone .publishing .a paper containing a com- 
putational comparison of codes "should make those 
codes available ^0 all other researchers, and 
perhaps even a cihtral -facility shou.ld be created 
for storing them. 

■ • * ■ . . ' 

HWGREERBERG: '.I think the question that h^s now . 
been brought to the floor is not -guidelines for • 
• running tests,' but guidelines for publicatioa of 

>^ tests. We' are narrowing the subject to just the • 
issue of publication criteria where the contribu- 
tion hinges^on- the claim of a better algorithm.- • ' 
One of the proposals ^then is that rid one sh6uld be ' 
allowed tb publish results claiming -better ^jerfor- 
. mance of. an algorithm on' a set of ^problems* that 
axff not in the public "demain. More specifically, 
wha^ver problems have been used by researchers. 
must\be made available either for. the sake of - 
reproouQability. by referees or for other research- 
ers to tr^^^-etft^heir methods on tlie same set of 
problems. In sfibct then, it's the publication 
issue that's been put. on the table, not the - 

. general issue .of guidelines: ' . . . 

O'l^EILL : Can 1 ^confound the problem? How . 
about providing the code itself?- - . • • * 

R. DEMBO : I suggest that we attack the eas'ier ? . ' 
things first before we get on to the trickier j^p 
problems, like' exactly which -guideline^ you- woifTi-- 
suggest for publication. There are certain stan- 
dards that could be implemented immediately. For 

" -example, reproducibility: a referee should be con- 
vinced that ah experimefitjcould be reproduced. V . 
That's example of what I think is aji important" 
criterion. ' * ^ 

^ - ^ " • - ' 
R. O'NEILL : I could add that T think all of the 
important controllable parameters should be in- 

■ eluded like machine, comiDt^ler, operating system. - ' 
It is not necessary to piTe up the. paper, but 
this could be included 'in an appendix: 

H, 6REENBERG : I-'d like to point out that- in the ' 
early days of the develt)pment of the^5IMPLEX 
msthod, if very severe'^'restrictions iThad been -4. 
placed on the publication of results, it may ' ' 
never have gotten published. 



y J- GILSINN : In these "days of page limits per 

article, trying to include a listing of a.computer 
^ode is -.probably not a realizable idea, but per- 
hpas ft is possible' to require that enough back- 
ground information be included in such an article 
that- would then allow reproducibility if one 
. established contact with an author. Another ' 
choice would be. to provnde some central reposi- v ' 
tories. where the codes/could be obtained-v But I 
think there is a conflict here that' must Be ;. 
resolved. ■ . ■ . L\ ^ ^ 
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andMt is not possible to include'much information 
about serious large scale systems, 

» -*■■. ' 

j;- 6ILSINN : Yes, but I also want something more ; 
than just so^and so's algorithm. ,1 want informa- 
tion about, hovihat algorithm was implemented. I 
want to know, informatioa about what rjesearch tech- 
hiquejs Were'bserfkv v 



WILLIAMS : Both problems have 




handled 
statement 
A number 



simply by having the author incl 
that a listing of his cckie is available* 
of thfr-joumaTs are encouraging authors- to do this 
kind of thing. • 



H. 6REENBERG : What do you do then about proprietary 
systems where' it's informative to get into a journal , 
some analysis hut where the owners, of that systenr 
are not going to release ^their proprietary Infarma- 
-tion about the code^n the form that you're dis- 
cussing* Most certainly', they 're not going to make ^ 
available source listings of such systems as MPSX. 
Can we eliminate that frcm the journals? 

A, WILLIAMS : In my opinion, seriously, we wouldn'^t 
eliminate 1t^ but, we would charge them an advertising-/^ 

fee/-- . ■ ■ 

H. 6REENBERG : I think it's not a binary situation. 
I think that there is much useful" information that 
can be contained in an article without requiring' . " 
'.that a listing be associated with it. ^ 

R. D^MBO : When I origijja^ly- jnentioneci this, I 
didn' t have in mind a requirement that authors^ 
submit listiTigs. 'My ^oncern was. with requiring ' * 
♦ referees somehow to convitice themselves that given 
a listing, they could reproduce the results.^ they . 
had to convince themsejves of^ the integrity of the 
authors/ • ■ - .* 

H, 'GREENBERG : T don *^t think 'that integrity is the 
issue. 1 think the transmittal^ of information is 
the issue'i. ^ . .. * 

E. HELLERMAN :' In this connection, what worries me 
is that when comparison^ are made, they're made . 
against your code and someone else's code* Very 
frequently^ it*s someone else's code that J« at 
stake^nd in this case we don't know what kind 
of . implementation that person has of that other 
person's- cede. This is worrisome. 

■ ■ . ■ X ■■ " ■ . - 

R: JACKSON : I. think we should keep "in mind the 
purpose for requiring additional information about 
algorithm or<:ode- We're seeing lately that^it is' 
no longer sufficient to run a few problems on two 
codes and report ovei^all CPU times which are then 
used to support claims of superiority. It is be- 
coming more and more important to understand what 
is going on in the interior of the codes that are 
being compared. Knowledge of pivot strategies, 
for example, is important. And if you deal with 
proprietary codes then there is no satisfactory 
way to get information aboiJt what kinds of tech- 
'niques are used in the algorithm of which ^that 
code is an implementation. In this case then,"' 
we're left in a situation where that code simply 
cannot be compared^ ^ 



able because we're testing codes, not algorithms. 
I think we've thrown about the words algor-ithm, 
code and software, a little haphazardly land that 
in our eScperiments we deal with codes and jnake 
inferences about algorithms. ^So that, if we're 
going to be scientists, those codes, should be 
available. -^However^^ I don't think that is a. feas- 
.ible.suggestion right ,nbw. There will be peopl^e 
who are unwilling to ijiake their codes available* 
and^that would just shut, off what fearful little 
resiifts we have now; at least reduce it quite- a< ^ 
bit. I think we 'can work slowly, toward requiring 
more and better information to be provided either 
through an appendix, or by way of the authors • 
themselves. Our problem then, is to determine 
how to increase the informational .flow. ' ^ 

H/ GREENBERG : I think it has to be acknowledged 
though, that there is a trade-off. The goal is 
to increase, the transmittal of useful information 
but i'f you impose restrictions saying "You will 
report this", that doesn't necessarily result in 
an increase of information because the reply can 
be "No, I won't.-" - . ; ^ 

R.. JACKSON : The situation then becomes what Al 
suggested where authors should be charged for • , 
advertising. Papers would be appearing that are. 
essentially "a claim of superiprijty with no sup-, 
porting information allowing a, replication of ^ ^j,;, 
t^e experiment, or even the checking of the types' 
of stra,tegies and techniques used within a code, 
r.m -not^ saying that is notj^ transmitting more ; 
information. ^I'm simply saying as a cod^ com- 
parison or a claim of superiority, there is* much 
room, for improvement. . . ' ■ . * 

J> MULVEY :.- I think there's an analogy to be drawn 
:in consumer unions in the sense that they go out 
=and test and break, and produce good results on-*, 
that empirical evidence. In our case we .could ^ 
haVe-a group thitgoes to. a particular installa- 
tion with-a battery of test problems and says 
"Herej^:solve it on your ma.chine,^ on your algo- 
ritttnC etc." ^ The big question fST^yWho'^-. going 
to do that- woVk? ' T^hat's hot c3 ear. 
- *" ' . ' " ■ • i ■ . . ■ 

E^^ HELLERMAN : . I think a bigger* question 'is: \ 
who*s goi ng to* pay for it? . . 

C. -MYLANDER : The suggestfon that ought to be^put 
forward is that requirements- f on publication ought 
to be that the 'test problems and the houi?^ that, the 
.test was run are fully specified., I don't^lihink 
that it's necessary to specify, the codes also. In 
this way, if I have a code to solve a class of 
problems, I could run on exactly the same test 
problem if I wanted to go out and rent time on 
that same computer and run 'under the same opera- 
ting system so that a valid comparison could be 
made. ' , 

J. -MULVEY : I think he used^ an adjective that 
forces you into the situation of seeding out codes 
and that was that the experiment must'J^e Jlilly. 
described^ Fully described means that. you have ^ 
to have a code^ You can summarize it, certainly, 
but you will be losing irrformation. 
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UNKNOWN : It seems to me that we have" two different 
■ items here; algorithms and codes, and that they' re 
completely different. If someone wants to say "I 
have a code and it!s a good one but I don't want to 
tell you what's in it" that's fine, so long as hg 
presents the problems that he, ran and presents his- 
tests along with, the environment ^in which he ran 
it. If it's an algbrfthm he's pushing, it*s a 
different story. He^tnust describe_the algorithm 
behind the-code in detail to make it valid, and he 
must also provide a listing. 

H, GREENB£R6 : I think a good example is the 
Hellerman-Rarick invert routine, where' the invert 
code was not made available for free, but it was 
nevertheless a valuable contribution to the litera- 
ture:'to have l^r* published- ;' . . 

A: WILLIAMS : Why, though, woul<i something like 
that be published as a scientific paper rather 
than as an advertisement or*flyer? 

H, GREENBERG : I don't think the name . "Management 
Science Systems" meant anything to sjDmeone reading 
the article-so I don't think- it was an advertise- 
ment. ■ I think rather it was a major contribution 
to understanding how reinversion should be done, 

A, WILLIAMS : I'm not familiar with the details. 
Did they speak broatily about what was done, not 
specifical ly? ■ . . ■ 

H. GREENBERG : No, they gave specific algorithms 
but the implementation of the algorithm and the , 
■code was not specified and that makes a big dif- 
ference when you^try to duplicate, 

A, WILLIAMS : I guess I don't understand. When 
there *s some kind of a mathematical, procedure, 
i>nless you knew the real trick when coding it, 
you me.'^n you could net code it? . " 

H. GREENBERG : No, you could code it, I coded it. . 
But you almost certainly don't have as good an 
implementation as the one Dennis Rarick had. So 
that if you tried to compare MPSX's invert rcutine 
with your.home^rown code, you won't necessarily 
be comparing like items unless you're as skillG(S 
a programmer as Dennis Rarick wasi Very few . 
people are. Nevertheless, the whole algorithm, 
is there and there's enough informat>on far you 
^to code it. On the other hand,, it's not just an 
algorithm. It offered whole new concept Sf 
looking at rein version. The idea of a spike wa^ 
introduced in that paper which has become a class- 
ical paper in the literature of mathematical / - 
prografiming systems. Much subsequent research 
has "taken place because of it. * • 

J. MULVEY : Why does the company allow you to ' 
publish an algorithm which gives the. essence of - 
an idea and not publish the software- It seems 
to me that both the idea and the implementation 
are proprietary, . . 

H... GREENBERG : It's- for precisely that reason. 
Because the full power of -the method rests upoir " 
such clever coding that they were unafraid of ^ 
the competition and atlpwe'd it to -be published.' 



E. HELLERMAN : Actually, PS.was described in terms 
of ALGOL. There were ALGOL-like statements descri- 
bing every facet there.* Also, I've gotten reports 
from a" number'of^ universities stating that they 
implemented tl9fe al gori thm strictly on the basis of" 
what we had written in that report, P3. It was - 
clear cut thjat they could write their own code 
and have^t working almost immediately. 

M. GDTTERMAN : Actually the code itself, is depen- 
den.t not only on the machine, but on the data 
structure with which you happen 'to choose to rep- 
resent yo^ur LP matrix and ^P inverts. 

UriKNOWN : We*re actually introducing a third prob- 
lem here because the code structure is another 
dimension completely. 

H, GREENBERG : Absolutely.- You run into that all' 
the time where'a major ■revision" in the data struc-y 
ture has a greater ijnpact on the resulting per- 
•formance of a code than a major change in the 
algorilJhm tactics. 

H.' GUTTERMAN : I can. testify to that one personally 
since I'-ve been involved in looking, at the results 
of three separate implementations of the KaTan 
Matrix Packing Ideas ^ the same computer. 

R. JACKSON: Perhaps we're getting to the pdint ^■ 
-where we ought to broaden our definition of what/" 
exactly^ an algorithm is. . " 

R. ^DEMBO : No. I think my original comments re- 
ferred^not to whether an algorithm was pubTished 
'in 'coded form or not. I agree ^^h the comment, 
by the wayr that if you publish^e code it would . 
mean less than publishing a mathematical descrip- 
tion. But let's take the paper P3 as an example. 
If claims were made in that paper that the new 
P3 invert j)rocedure was much better than what had 
been done previously, the question then becomes V 
whether enough information was given in that paper- 
to allow those claims to be- tested by someone else 
willing .to code the invert procedure. It must be . 
understood that these conrnents- are made with the 
understanding that this new researcher might be a 
worse programmer than Dennis Rarick was. 

■ J. GILSINN: I'd lik^.to go back to a previous,, 
topic of conversation. Are you thinking in this": ' ^ 
set of guidelines of not allowing a paper to be "■: 
published if it presents computational results 
about a code that^ is a -proprietary one? Becaus^e 
it seems to me that users of codes are m&s^ inter- 
ested in knowing the performance of a particular 
code against one of the better known codes^and 
very often the^glDettericnqwn codes- are t'he 
proprietary ones!? Whose go^ng to restrict that 
kind of a' situation? ' 

• H. GREENBERG: I'think the correct answer is that 
we don* t know. . „ ; * ■ / 

R. JACKSON : The issue is getting a little more 
complicated,, and I'm not sure we're defining our 
terms very well. Our origthal topic- of conversa- 
tion was the development of a. set.of guidelines * " 
..for the publication of ..comp^itational /results whpre 
the ultimate aim is to see better cdmpartsons 'made.. . 
We went off into the tangent of discussingi^het^r 
the code itself should'be publ ished as! a result. 



of the comment that if you don't see the code you 
can't do a proper comparison. Another issue has 
been raised, however, that in the absence of com- 
plete listings of the code for whatever reason,, 
proprietariness or unwil 1 ingne^ss of journal editors 
to include, listings , is there anyway that a fair 
computational comparison can -be made about codes? 
I would like to see that topic discussed moreso 
than the current topic of whether codes should be 
published or not. It is clear that proprietary 
codes are not going to be published. 

W- ORCHARD-HAYES : But related to that question is 
the argument that there is nc^such thing as repro- 
ducable complicated coding. -^It depends on too 
many factors including the^^tyle of the author, 
th^ compiler you're using and whatnot. 

R, DEHBQ : . I didn't mean repri^ucing Code, I meant 
reproducing experiments . 

W, ORCHARD-HAYES : No, I inean reproducing results. 

R. DEMBO : Well, I think you're right. There are 
a lot of, dii'ferent factors that- enter into it. but 
you should be able to attack the same problems, 
uising the^same tolerante criteria as the authors 
did. and. produce the same results*. 

' W. ^ORCHARD-HAYES :; I guess the way I'rr --.i^.utj ' ■ . 
it's a QuestioTT of relevance.. What di rf erence "does 
'it make? 

R, JACKSOM : The answer to that >-^vc'ves ;nd 
.."s* fhe question of replica'tion as a le: "sarv part 
;.of'scienti/ic endeavor.. ■ 

• R. DEMBO : The point is that you re making infer- 
. ences . 

H. GREEraERG : ;I thinl- the payoff '*s a bit better \ 
^ . than that. now. - I think it's hard t^:^ understand 
•' when there are no controls.* If one. ;:>eVson runs ' 
ah experiment with" one problem and soirteone else 

■ / runs another experi^neht with another proi'lem, 

' t&re. is no way of knowing whether' you've r.v;de 

any progress. " I think the value of -reproducibi I i ty. 
: to the extent tJiat.it's feasible, is -that it does 

• give you a measure^ of progress. - 

J. MULVEY : " I think there is some value in- saying 
why one code is better than another in that when 
someone else tries to duplicate the experiment 
they can at least haye some, systematic way of 
trying to get t^he same kind of results. 

H. GREENBERG : f think it's* true for the opposite, 
reason, namely to inject the objectivity that make 
it less dependenton the judges. I think that's 
the poi«t of reproducibility. - . . / 

^. ■■ ^ ; . ■ ' : - • r ■ ^ . /; 

. E. HELLERJ4AN :' I- think there's an awful lot of pure 
■ artistry here\ artistry in implementation.. Now, I 

would stack , any thing th*at^ Bill* writes against any-^ 

■ . thin'^.thiat anybody else writes,, and I know it's . .. 
•^going jto be /better because'^Bill * s an artist at. 

this kind of th,ir1g. It's; Tike looking .at a painting 
of the same' landscape by -two ^different artists. . 
TheyVre goijtg to Jook a little different no matter 

• how hard they try to'fae *the sam^. A don' t "know if 

• you xan develop. a criteria for pitting one against 
another to determine which one is better. 



R. JACKSON : Lthink I would like to disagTee With 
you on that. I agree with you that there's an 
incredible amount of artistry in producing sorpe ^ 
of" these codes, but artistry,- I think, once it is 
around long enough and used over and over again • 
.is no longer, artistry. It becomes documented ■ 
fact. Let's take list structures as an example. 
At some point in time years ago, how data were 
stored, organized and retrieved was almost a* 
black art. But now techniques for organising data 
and retrieving it appear in textbooks. And this 
is getting back to what I mentioned awhile ago, 
that it might be necessary to expand our defini- 
tion of an algorithm to include such items as 
specific coding, tricks. , ■ 

W. ORCHARD-HAYES : .Why riot^just say "list/structure" 
then? Why isn't that sufficient? 

.:.- R,- JACKSON- :- My point here- is~that -there are. 
probably other aspects of what is" now artistry 
that should be included in this expanded'defi ni- 
tion of an algorithm. 

H. GREENBERG : I agree that what was art .ten years 
ago may be partially science now. . * -r' 

■ W. ORCHARD-HAYES : I'd, like to make one more point 
about comparing codes and that point has to do'vith 

-portability. I just can't believe^that it's pos- 
sible to carry one code from one machine to another 
machine and compare it.* Codes just aren't that 
;>ortabTe anymore. 

H. jREENBERG : Right. We have identified machine 
Che 'acten sties as a set of- variables to be reck- 
oner with in the design of an' experiment, r think 
thrrugh scientific investigation we can get a much 
bet-er understanding of the various computer and; 
alcorithmrc effects on the results of -an evalua-"' 
, ticn. That, of course, is the point of ' the inves- 
tication: to understand the effects of the TOri-* 
ar-^es'^ .. It's r^y feeling that there will be somQ||^ 
"-'nteresting results from^the work done' recentl^F - 
by Dick O'Neill^ L believe there wilT be some 
very strong and counter-.intui tive results from 
that work. 

M. GUTTERMAN : I personally don't believe in prin- 
ted publication of codes, I think that code 
avaiVability by publication, the creation of a 
collection and dissemination center foY* machine 
readable code, is a .valuetble con^^bution in^:5tSTiy 
cases. But despite the artistry "of Bill Orchard- . 
Hayes's coding, ''I ' ve never gotten'much benefit 
from having a listing of it in front of me. 

" " R. O'NEILL : You; know, if you chose, the test prob-^ 
lems correctly.^ if ou could actually design an experi- 
ment to test onei'person's artistry in cpding against 
another's/ You could in fact make inferences about 
artistry. . .. : . 

: UNKfJOWN: This'sort of [thing is ben ng done and'.re- 
iuTTFlfarve been reportgfd in a book called The 
Psycho>ogy^ of !CbmputeV i^'rogrammi ng .- The exj^eriments 
were designed ^i>:xest whether' a code .produced -by : * 
different people varfed' accOrdi^^g to the stated ^ • 
goals. . It made a tremendous difference Whether 
the stated goals :were to produce' coda as quickly ' 
as possible' or whether^the code was meant tb^be'^ 
as effi'cient as possi'ble. In any event, this kind 



• of experiment is being done and the results are 
very interesting. 

• E. - HELLERWAN : ' I'd like to point out something here 
about characteristics of computer programming. I • 
actually recall an instance where one of the oil 
companies was approached by Bonner ^nd Moore afid 
informed that their problems could be running. in 
one third the time' that they're currently taking. 
Bonner came to. our installation to run it and 
spent a lot of time fiddling around with the formu- 
lation of the problem until he finally met his 
objective of solving the problem in one third the 
time that it. was currently taking for that oil 
company. But tne reason that he was successful 

is that -he knew enough about the characteristics 
of his code and what is required in the way of the • 
characteristics of the formulation, in order to 
take maximum advantage of his cod'e characteristics. 
So these are a few other things-one-runs-up- against 
when tryi ng >to measure code performance. 

M. 6UTTERMAN : • Another question to be answered 
however, is what would have happened if they had 
.taken the new formulation of the problem and put ' - 
it on LP90 -f would it have run* faster in that 
case? , 

• E. •HELiERfWT : Another important point though, is-- 
that Bonner Icnew enough about the oil company's 
problem to say- that if you can live wi th an error 
larger than 'was being allowed by. LP90, you can get .*: 
the kind of reduction in" time that *he. was talking ■ " 
about. And he indeed came up with absolution that ' 
had an error at the maximum rate, a rate that would 
never have been colerated-by- LP90. The question ' ' 
then becomes what can you live with?* How large 

an error are you.willing to tolerate? 

• • - . . . • * " . ' ' ' ' 

R.. JACKSON: ' That question o^ what you cau live > 
: witftris a difficult oi:e and rftakes me think of the 
conrntent Chuck Mylande'r made earlier about wanting 

• .to Keep only a snrall section of codes. There' 
are people that;, I' ve Sone work for who want only* 

_ one nonlinear programmiHg code. One man* doesn't 
care whether there ""s a few seconds difference 
between ^he code that he's got and the other codes 
that he might be able. to use. He wants only to 
learn how to operate and became familiar with one 
code rather than have ta deal with a large number 
of them. ' * 

• ; C MYLANDER : As a referee, I still get alot of ' 

papers in which an algorithm is proposed but the 
paper doesn't have , any other merit than the pro^ 
posed algorithm and it hasn't even been coded.-. ^ 

• H. GREENBEi^G : Why don't you reject it? 

; " Cu MYLANPg^ I do^ It's juiit that I-think-it- ' 
would be*' a . good idea to proj)ose a standard- for - 
people who publish algorithms a"nd computational 
results making it, mandatory that they have run 

:■ the code, .specified some test- problems, and 

^listed the environment in which it Van. 

. H, GREE^SERS : The evolution I see, for example' 

• tn nonl i near programmi ng, is that fi'fteen or ' 

■ twenty years ago we were pretty hungry for'al-' I 
go r\t hms- making- it possible for an algorithm to I 

,be proposed with no mgre justification other tha\ " 
that it. was a clever new idea.. As time w^nt on, \ 



however, it became necessary to provide empirical 
evidence, or theoretical evidence like rate analy- 
sis* about the efficiency of a clever new idea. 
At that time, say in^the sixties* it was acceptable 
to do an experiment consisting, completely .of ^ * 
randomly generated problem. In the seventies, ".■ 
however, we*re discovering that is no longer accep- 
table and there appears to be a move toward, much 
more sophisticated design of experiments to test 
whether an assertion of a superior algorithm "is 
•true. : ' ' 

H. CROWDER : What's the possibility of sending 
programs and listings to referees that can be 
used. in refereeing the paper. For example,"! 
once was asked to referee a paper for TOMS. The 
editor. Milt Gutterman,.vsent rife ^ deck, and a 
listing, and the third example I' tried on the code 
..failed. I simply packed it ail up and sent if 
baclj: to the editor . and it*s back ,in the hands of 
the author now. ; : 

R. DEMBO : When I originally brought the^. topic " 
up,M was thinking about the kind of article that's 
published Th which the authors say "Here is a new 
algorithm that took two seconds to solve Rosen- 

• brock's function and therefore this is a really 
neat new algorithm. That kind of article is still 
being published, for example, in Mathematical 
Programming: It shouTdn' t be. We are now at the 
point where we need much more information than, 
that it took two seconds to solve Rosenbrock's 

/FCinction. . . 

UNKNOVJN: ^ One thjng I haVan' t .heard mentioned " 
today is the situation where -an algorithm-is pro- • 
posed to solve a previoully unsolved- problem or 
"for some other reason there are , no other algo- 
rithms-to compare against it. My question is : ^ 
should the proposer of the algorithn^ be required *. x a 
to program it? There may be a number of math- - /- 
ematicians who simply are unwilling to do. th'at^- 

H. ■ GREENBtRG:- As^I understand' the spirit of the 
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gu1delln€s\ they're based on the assumption that 
they will be*^pftlied only^ for papers whose primary 
.contribution is based on a claim of superiority 
or some other way related to competition with 
other algorithms for solving problems. Only in .* 
> that circumstance are the guidelines applied and 
the authors are asked to satisfy certain experi- 
mental design criteria. Not much thought has 
been -given to the situation you' re describing. 

R. O'NBILL : Before the session ends, I would like 
to ask the audience a'^feW Questions about some, of . 
the topics we've discussed today. I'd like to. 
know, .for examp-le, how many peopTe*here think that 
■'the computer hardware should be mentioned or spe-. 
cifically included in a paper to be published 
about computational results? What about the 
operating system? The compiler? The specific 
test problems used? Should the test problems.be ' 
made ava,j'lab1e? I thipk we have a consensus. 

H. GREENBERG : One more question. How many care 
^whether the publications carry that kind of—., 
.information? 3 

R. DEMBO r.'"^ Lt afipears', gentlemen,': that we. have . 
Won. . \' ■ *. ' • 
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R, O'NEILL : One more question - how about the < 
code? How many people think the code should- be 
made available? No consensus. What about avail- 
able,, but not publi shea? No consensus. What. . 
about available to the referees only, for evalua- 
tion?. A consensus. ■ J 

M..GUTTERMAN ; That's a suitable :c(»npronrise. 

R; O'.NEILL; What-about proprietary codes?' ^Should 
they be made available to the referees only for 
evaluation? No consensus on that topic. 

R. -DEMBO; We've asked a lot of questions here and 
got. some very good input from the audience. The 
next question we have is what exactly should we do 
with it? How do we^go about getting these ideas . 
implemented? 



J. FILLIBEN : 
lal boards? 



Isn't' that a problem for the' editor? 



H, GREHNBERG : Yes. As I understand itjjfe should 
next send a: letter to Michelle Baliniski T the editor 
of Ma th ema t i ca 1 P r oq rammi n q , with copies to editors 
, of other journals of our field. . Since we're a 
Gonmittee of the Mathematical Programming Society, 
and since Balinski has already indicated a great 
desire to have us produce these' guidelines, we can 
.expect favorable treatment. 

JACKisON : It!s time, now jto , sum up"^ this .session', 
. since we have run out of time. I would like to 
poin.t out that the next step for the Committee op 
Algorithms MS to draft a set of proposed guidelines. 
We plan a panel discussion at -theVSan Francisco 
meeting of ORSA in the spring of next year where 
these guidelines will be discussed by seven asso^ * 
ciatfi. editors of the journals of our field. After 
that, a final version, of the guidelines will be 
sent to^he editors cf those journals. The comit- 
tee also is. organizing a research exchange, or news- 
Tetter- to keep informed those persons who are 
intereste:- "ir the work of the committee. We will 
be compilir^'- a mailing list of "friends of the 
corrmlttee", and if anyone is interested please 
send your name and address to any member- of the 
committee. With that, we can end the sessipn by ; 
saying thank you all for coming. 
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ABSTRACT 

Thevconcept of. search enumeration in integer 
■ programming Is applied to, find the optimal 

> schedule in a plastic injection industrial • * 
process. The production process involves 

, molding parts 'using a molding machine which 
;usu^lly has several cavities. ^ Each cavity 

* accomodates a die which makes, a single part, . ^ 
Given a set of customer- orders which require- 
certain production days, (known), the problem 
Is to find a schedule which -minimizes the nxm>- 
her of molding maci£ine setups, while satisfyr 
Ing all te^rhnologlcai and logistical con^ ^ V 
straints-" This paper presents a search 
enumeration algorithm to find an optimal^ 
schedule. The- algorithm is folloyed'by an • 
Illustrative example. Some compu tat lonaT 
results are mentioned,- and other comments, 
concerning algorithm improvements arc also 
glvisin. 

All work discussed here relates '-to part of:^an*; 
actual case study for a medium size Norther t . 
Ohio Corpojration. The entire case stxidy 

* resulted in the development of a computer 

* system for production scheduling as well as* 
■for due date assignment. Inventory control, 

> machine allocation, and extensive data 

. processing. In this article we concentrate 
our efforts on the optimization phase and 
" its Implementation for the production' 
"scheduling part of the system. 



*No part of this document may.be reproduced with- 
out explicit written permission of both authors i 



1. INTRODUCTION 

The concept of search enxmeration in Integer 
programming is applied to find the optimal 
schedule In a plastic in jection~3jadus trial 

'process. The production process Involves 
molding ' parts using a molding machine which 
usually has several cavities. Each cavity 
accomodates a die which makes a single part. 
Given a set of customer orders which re- 

. quire certain production days (known), 
the problem is to find a schedule which - 
minimizes the number of molding machine 
setups, while satisfying all technological 
and logistical constraints. This paper 
presents a search enumeration algorithm to * . 
find an optimal schedule. The algorithm 
Is followed by an illustrative example. 
Some computational results are ^mentioned, 
and other comments concerning algorithm " 
improvements are also given. 

All work discussed here relates to part 
of an actual case study for a medium^ 
size Northeast Ohio Corporation. The 
~ent Ire case study resulted in the develop— 
menz of a computer system for production 
"schedul;tag'^ well as for due. date assign- " 
mcnt , Mventory control, machine allocation, 
and extensive data processing. In this - .<:^ 
article, we concentrate our efforts on the 
optimization phase and its implem^tation 
for the production scheduling part 'of the 
syst^. '-.^ ' ' ♦ . ■ ' * 



2. A BRIEF DESCRIPTION OF THE PRODUCTION 
PROCESS 

In this section, we describe the production 
process which Involves molding plastic pairts ., 
via a molding- machine . A molding machine 
accomodates seve:;al (usually 6 or 8) dieis, . 
each of whlch^, corresponds to a specified 
part* Dies in a machine may be all different,' 
but certain technological restrictions exist. 
Most importantly^ these are die position' '• 
constraints, and die length constraints. . That 
Is, jdue to their geometric . chara^eristiics 
and ease of removal from .the die^! .^certain ^ 
parts must be produced by dies located in a 
mold position near the machine operator. These 
dies are said to require. a- "front position" 



In- the mold, and are labelled "front runners." 
In contrast, the reiQalning dies can have ei- 
ther a front or rear posicion» where a rear 
position is ac a farther distance from the oper- 
ator. In addition 'to front or rear die posi- 
tions » quality control dictates that some 
dies must be located closer to the^ center of 
the mold. ,Alsp, to avoid part breakage 
during their removal from the machine, the 
difference of lengths of adjacent d±4s can 
be no more than 3/A". 

It is undesirable and costly to stop the 
molding machine during a production run. 
Thus the molding dperation is basically a 
continuous production process. New jobs 
to be processed correspond to orders for 
distinct parts. Based on an order, quantity , 
the number of production days required is 
obtained. At the end of production for a 
certain jo]b, a die must be pulled cut of a*" 
cavity in the molding machine and ^ new die 
required for the next job is inserted. 
During this setup » the whole molding machine 
must be stopped, which means that production 
is lost* Although a setup can be completed 
fairly quickly, say in 30 minutes, it could 
take se'Veral hours until production stabilizes 
and acceptable parts are molded. Therefore,, 
in order- to maximize production, it is desired 
to minimize the number of setups. 

Even though our actiial problem Involves mul- 
tiple plants and multiple machines > as well 
as some other constraints (such as mold speed, 
plant specification, etc.) ,* In order to * 
^simplify the. discussion, we concentrate on : 
a 'one^machlne example. We also asswe that 
orders <:o]p:espond to distinct parts, only one 
die is available for each' order and the sched- 
uling period is known. • (It Is actually de- 
termined ' from a previous model in the sys- 
tem.) Our goal is. to find a schedule, which 
minimizes* the number of setups, while, satis- 
fying all technological constraints- The ; ^ » 
next section. describes a search enumeration . 
algorithm wiiich finds cm optimal: schedule- _ 
• " ' - ' , ■ " '. 

An Iliustrative Example (1 machine, 11 orders) 
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Number 


Production 
' Days 


Die 
Position 


Die ^Length 
(inch) 


1 . 
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-Front 
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Rear 


2.5 


r ^ 


3 ^ 


Front 


3.2 


V 4 
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Rear 


2.8 
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3 


Front 


3.6' 
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Front 


■ "3.3 
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3 


Front - 


2.9 ' 


* 8 
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Rear 


'2.8 
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3 


Front . 


3.0 


10 




Rear 


3.0 


11 


.4 


•"Hear , 


2.7 



The initial conditions of the machine schedule"" 
(usually due to the previous schedule) and 
an arbitrary schedule which satisfies the 
techn'ol^ical constraints is shown in Figure 1. 
Ic^ the figure, "the shaded area indicates the 
iaitlal conditions and the numbers in the 
shaded blocks are the die lengths,. The 



scheduling period is assumed to be 5 days, 
and the search algorithm discussed in the/next. 
section seeks an optimal schedule which flll^s 
up this scheduling. period with' the smallest , 
number of additional setups. 



Die ?o«lrion 
Front Re«r- 




•Front runper 
^^Addltiooal sccups' 



: FIGURE I. Initial^ Conditions (shaded area) & 
an Arbitrary Schedule for the 
Illustrative Example . , 

3. FORMULATION OF. THE PBiSbLEM - , " • * 

The algorithm is similar to a branch and bound 
approach used in scheduling theory (e;g., see 
Baker [T]), ^ut 'the nature of the problem, as - 
explained^ shortly, suggests a search enumera- 
tion used" in'. Integer programming (e.g.,' see 
, Saikin [2]).* As in any entimeration algorithm, 

*A^search enumeration, as opposed to a branch' 
and< bound enumeration, was used because of the 
following: • " ' . ' 

^ 1) A branch and" bClind scheme tends to create 
many dangling nodes. That is, those that, 
have not yet been considered for branching 
. p'^ rposes . This may cause < computer stor- , 
age difficxxlties; whereas, a search enu- 
meration deals with^only one node at a 
tiine, at ■ the cost of more bookkeeping, and 
does not require the storage -of i:a set of . 
dangling nodes (see, Salkin'[23). 
Saving computer* storage, was especially 
important t-OL„us because the system is 
being implemented on a^small computer 
(64K bytes of joain core) . 

2) A clever bookkeeping scheme, based on ^ 
. the fact that a forward step corresponds 

to locating a die. in. a position with an 
eaxilest calendar opening, improves the 
performance of a' search enumeration and" 
Uses a very minimal amount of storagel 

3) A search enumeration always gives a 
current best feasible schedule and 
xisxially produces a .near • optimal sched— 

. ule very quickly (see, Saikin [Z]). . ' 



th6 procesa Involves, branching and bounding » 
and can be plctorlally represented by a tree 
consisting of nodes and branches. The nodes 

" correspond to dubprobleias and the branches- 
link subprobleas i^ch differ by a single 
additional Job fixed. in a die position. Ve 

'nov discuss the procedure in contexit* of an 
enuaeration tree. ' 

Let P^, the initial node in the tree (see 
Figure 2) , denote the problem containing n 



Jobs. 



^0 



The problem P can be partitioned 

III . . 

into n subproblemsy' Pi ^ • • » P 

, signing jobs to the fdLrst opening (l^e^t 
subsequent to forward steps) .By "first 
y openlxig'* ve joean the die position which has 
\ . ^' the ea riles t^calendar opening . In case of 
\'. ties y .ve can use any consistent rule such 

\aa the smallest- index. Thus, is the ori- 
ginal problem with job 1 fixed at the ear^ 

. liest opeiiing; .P2 is the original problem 

with job 2 fixed at the ekrliest opening 
. etc. In the Illustrative Example , the ear- 
liest ^opening is day I at positions I ''and 

.5 i ' Pi is then the original problem with job 
• 1 assigned^.at position* 1. (We are using the . 
smallest ' index rule, in case of ties.) Next, 
each of the aubproblems. can be further par-; 

titio'ned as In Figure 2. For instance. Po 

' 2 2 .2 

can be partitioned into P2it ^Z3'* *^2n, T6t 

, • 2 • . • . 

'. example y ln P2ii Jobs 2 and 1 are assigned-' 
at 'the first two earliest openings in this, 
order. In general, at level ..k, each sub- 
problem contaixis. k jobs already scheduled 
(or fixed in.. a die position)* Each sub- •"* ' 
problem>can ;be further partitioned into "at : 
. . most" (n-k) subproblems, which form part of 
■ ;.. ; the level (k^-l) subproblems. The reason 

- why-t^c say "at most" is because of the tech- 
nological constraints mentioned earlier, 
which may limlt^ the production of ^ome parts ^ 
at certain die positions; -Tor " example, 

• 2 . ~ ^ 
problem P21 does not exist in the Illustrative 
Exaa^le as Job 2; which Is a front nmner,- 
cannot be placed in die position 5 which is 
a rear 'die position- . 'This type of • implicit 
"enumeration may also 'be the result of the die .. 

^ length constraint. 

' i. , — • , _^ . " . ■ " - ■ 

We go down the tre§> until a schedule is 
'completed and a solution Is^ found. .Then a . 
backward step (a return to the previous sub- 
problem) is taken and a regular search prb-^ 
cedxire starts. The detail of a search . . 
enumeration' can be found in Salkln [2]. A 
part of a search tree for the Illustrative 
Exazoplej is given in the next section (Figure 
3). • • . 




Level 2, 



FIGURE 2. The General Enumeration Tree 



4. ^THE ALGORITHM 

Notation 'We denote the .set S as a partial se- 
quence of jobs -from among the n jobs origi- 
nally in the problem. * For example , S ^ , 
(2, 4,3,1), means that job 2^ dLs scheduled, at 
the earliest opening, and job 4" at the 2°*^ 
earliest opening, job 3 at- the earliest 
opening, and Job 1 at the 4^^ earliest opening. 
Also,.S7 - (2,4,3, 1>7 means S - (2,4,3,1,7), 
or that job '7 is scheduled at the 5^5?" ear-^ 
liest opening. . . ' 

Algorithm Listing ' - 

^ Step 1 (Initialization) 

Set 2*, the -current, smallest number of set- 
ups to an ' arb 1 tr ar ily large ;*value . S ^t k-0 . . 

The problem.'is P^ and S is null. . Go to.^Step 2. 

Step 2 ' \ .: 

Find the! (lc+-l)s^-earliest opening- die posi- " - 
tion. In -case of ties,- select the smallest " 
index; Go to Step 3: 

Step 3 (Forward Step) . . ^ ' . 

At the (tf-l)s? earliest opening die position> 
try. to. schedule- any one, say order i, of . • 
"untested" "orders (i.e. , exclude orders al- . 
ready tested or Scheduled) which satisfy the. 
die position (i.e., front/center/ rear) 
and die length constraints. 

=.kf 1 ' , ■ 



This defines problem P«. • 

' (A) If there is no^^such order," go to Step 4 
(Backward' Step) , Count the total -num- 
ber of setups thus.f«r, denoted as Z. 

(B) if 2'>^ 2*, mark' the order as "tested^ 
, (at the current earliest opening die 

.. * position) , and 'repeat the Forward Step 
with another "unteisted" order. - . 

(C) If 2 < 2*. and -the schedule is completed,, 
set 2**2 "(improved schedule found). 

. Set k-kfl and. S-Si. Go to Step 4. (Back- 
.ward Step)i . , ■ . ' 
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(D) If Z < 2* end the schedule is not yet 
cotapleted (Forward Step)* set k-krhl 
' and S*S1. Go to Step" 2. 

In any case^ an order Is never scheduled if 
its production duration exceeds the tine the 
partictilar molding machine is expected to 
remain operable* ' (This is determined from 
,a different model » not discussed in this ar- 
ticle.) 

Step__4 (Baclcward Step) 

Back up to the last scheduled order» and mark 
the order scheduled at the die position being*, 
opened as "tested." Iti other words, if 
-S"S'i (i.e., the last ^.scheduled order is-- 
order i), remove order 1 from the current 
schedule and return to problem P^7^- Set 

.k"k-l, and go to Step 3. 

Step 5 (Termination Test) 

If k < 0» stop; Z* is the minimum number of 
setups. Otherwise, , count the number of set- 
ups, z", fox the current schedule (i.e., S.'). 
If Z > Z*, go to Step 4 and take another. 
Backward Step. If Z < Z* (more precisely, 
if' Z * Z*-l), gQ to Step 3. • 



5. AN ILLUSTRATIVE EXAMPLE - 

We now describe how the algorithm works 
using the Illustrative Example, and assume 
that initially* an_ arbitrary schedule as shown. 
in Figure I Is given. This schedule is equl- - 
valent to P^^, where S - (1,2,3»4,5,6,7,8, 9, 
10,11), witn six (6) additional setups. v. 
(Note that setups due to the previotis schedule 
are not counted.) Therefore, Z*-6, and the 
j9t^5 below follow.' The tree corresponding 
to part of the computation is in Figtire 3. 

. Step .4 (Backward Stepj u.l^^ . \ ^ 

^ Remove ^rder. 11 from the current schedule. 
* Mark, order 11 ischeduled^ at the 11^^ earliest 
: opening die position (i.e., die position 7 ^ 
at day 4) as "tested." Set k«ll-l-10. 

Step 5 (Termination Test ) 

k ^ 0 and Z-6«»2*^6^ Therefore, go. to Step 
.4 and take o^r-lSore BacW^ard Step. 

Step 4 (Backward Step) 

Remove order 10*from the current schedule. 
Mark order 10 scheduled at the lO^h earliest 
6p enln g die position (i.e., die position 4 
at.day, 4) as "tested." Set k-10^1-9. 

Step 5 (Termination Testl ♦ 

Jc.| a and Z-6-Z*-6. " - 

Step 4 (Backward Step) 

Remove order .9 from the current schedule. 
Mark order 9 scheduled at the 9^^ earliest 
opening di? position (1. e;, position 2 at 
day. 4) as "tested." Set k-9-1-8. 

Step 5 (Termination Test) 



Step 3(D) (Forward Step) ^ , 

Schedule order 10 at the-d^^ earliest opening 
die position. Z-5 < Z*-6, and k-8+l"9, and 
S. - (1,2, 3,..., 8, 10). . 

Step 2 ■ ' . '■• • J 

The (k+1)^^ - 10^^ earliest opening (fie posl-* 
tion is die position 4 at day 4, 

:: Step 3(D) (Forward Step) • 

Schedule order 11 at the 10^^ earliest opening 
- position, ,2-5 < Z^6, and k-9+1-10, and 
S - (1,2,3-,*. /,8, 10,11). , 

Step 2 

The 11th earliest opening die position is 
die position 7 at day 4. 

Step 3(B) (Forward Step) . 

Order 9, front runner, cannot be scheduled 
at the rear, and therc^s no other alterna- 
tive. . * 

Step_4 (Backward Step) , 

"Semove order ll from the current schedule. ^ 
Mark order 11 scheduled af,' the 10^^ earlies^t" 

~ opening as "tested." Set k-10-1-9 and 
S - (1,2,3, -..,8, 10).^ . . 

Ste^5 - (Termination Test) 

k t 0 and Z-5-Z*-l. * 

Step 3 (B) (Forward Step ) 

Order 9 scheduled at the 10th earliest opening 
will make Z"6-Z*, and there is no other 
"untested" order at this opening i . • ^ 

Step 4 (backward Step) * " ' j^^y 

Remove order 10 from'^the<current schedule.** 
Mark order 10 scheduled at the 9^h earliest 
opening, as "tested*;". S'et k-9-1-8 and 

s ^ (i.2,3,-..,,8). r ; ' - 

StepJS (Termination Test.) ' * * ' 

k y 0 and Z-3-Z*-l . " " 



Then similar steps repeat by scheduling order 
11 at the 9^^ earliest opening without pro- 
ducing a better schedule, and. evenutally the 
enumeration reverts. tb^Ievel 8, level 7, 
etc. A part of the enumeration tree corres- 
ponding to the above description is given In . 
Figure 3- After we check all altemativlfe' 
at the IS opening die position, we have 
the optimal solution, which is shown In 
Figure 4.* Notice that 2*«3, 



k 0 and Z-5?«Z*-6. 
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6. COMPUTATIONAL SXPERIJENCE 

The algorithm has been coded in FORTRAN 
and la being Implemented on a small IBM 
computer with 64K bytes of main core. 
A few smaller problems have thus far been 
tested successfully*. * " • # 

' ' ' \ •> ■ 

Additional computational behavior corre- 
sponding to the Illustrative Example (Section 
5) is in Table 1. Running t*lmes relate- to' 
t^t runs on a UNIVAC 1108 computer* ; ' 

TABLE 1 ' 

Computational Performance for Example 1 



"Level k 



Time to reach level 
k first time after 
a Biackward Step '<s-e- 
conds/Univac 1108)' 



11 


0-000 


10 


0^001 


9 


. 0.005 


8 


0-011 


'7. 


0-015 


■6^- • 


O-O30 


• 5- 


0-034 ' 


4 


0.060 


- 3 


0,147 


2 


. 1,-290 


1^ 


r-509 


0 (Opti- 


3.984 . 


mality) 





Current smallest ' 
number of setups 



6 

6 ^ 

6 

6 

6 ' 
5 

' 5 

.4 
■ 3*' 
3 



.*The optimal solution*' with 3 setups is ob- ^ 
taineii after 1-404 seconds . 

" * ' ' "» . 

The algorithm can easily be ^tended to the 
multiple' plant and igultiple molding machine/case^ 
Hie actual case study involves many molding: 
machines located at several plants. Each 
molding machine corresponds to a particular 
mold type and to a particular material* and so 
the algorithm may be applied directly to all 
molding machines with the same mold/material 
type. ' The multiple plants normally add 
certain technological, logistical, and/or 
quality control C9nstraints. The resulting 
plant specif ication(s} naturally contributes 
to the algorithm* s efficiency. 

There are two different approaches when . . 

extending the algorithm to tHe multiple 
machine and multiple pl^nt case. One approach 
lays out 'all machines in parallel and ap- 
plies the algorithm directly. If we 'consider " 
a. two machine problem, each* machine with . ■ 
eight cavities, th^ approach reduces to ' . . 
considering a one machine problem with -six- 
teen cavities. Of course, the way to count 
the number of setups has to be modified, 
b^cAuse a setup in on& machine is independent 
of a setup in the other- On the other hand, 
a second approach to the multiple machine 
problem is to lay out all machines in series. 



In order to apply cfils- approach, a planning hori- 
zon has to be introduced, where the idea is to* 
schedule the machines during a given planning hor- 
izon- ■ An"" order Is scheduled only when , it can •• 
start" during the; planning horizon. . iSith this ap- 
proach, the algotithm;;, during forward steps, sche- 
dules the complete planning horizon of v the first 
machine, and then goes to the second machine, and 
so on. The backward stepi works in an exactly op- 
, posite way, . If this approach is adopted, then we 
^ are minimizing the number of setups whicK result 
from schedules that-„£lll up the planning horizon- 
Thjerefore, it i*3 possible that c^rtal^n orders cati- 
not be scheduled (or,' more* precisely, cannot be 6^ 
started*) .during the pijannirig horizon' in the optiV 
mal ^schedule (e.g., sec Figur«c 6). At this time,^ 
the program uses the latter approach.. 

It should also be mentioned that the algorithm 
currently does not allow for "a hole" in a sche- 
dule. In .other words, in a specific cavity of a 
molding machine, two subsequent orders- must, be 
scheduled in such a way that when the first one is 
finished, the other has t;o be. started immediately 

. subsequent ta a set up. jCt IS conceivable that by 
allowing a gap in a ' schedule. Cin practice, a cavi- 
ty-^s blocked in this case), the number .of setups- 
can be reduced.^ Computations with- a sample pro- 
blem is belpw^' In this' example , we have. 2 mach- 
li^es at two diCfement locations » A and B. Due to 

vcfertain logistical and other constraints not men-, 

r tioned, it turns out that some orders must be pro- 
duced at a. specific plant, and^thus we have a 
plant specification, Alsi, the two machines, de- 

^noted by X and Y have slightly different capabili- 
ties, and some orders must be processed by a par- 
ticular* machine (machine specification)- If no 
specification is made, any order can be processed 
by any machine. Machine X is located at Plant A, 
and Machine Y at Plant B. Initial conditions of 
the machine schedule and an initial arbitrary 
schedule Is given in Figure 5* The optimal sche-^ 
dule. for the Example, found by the search enumera- 
tion, -is' shown in Figure 6. Notice that orders? • 
6 and 8 are not scheduled during xhe current planr. 
ning horiron- 

EXAMPLE . 
(2 plants, 2 machines, 14 orders) 



Produc- 



*The material specification Is suggested in 
a previous model which is not disclosed in 
this article- The final material specif ica- 
tlotf is given by management. 
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The model is being tested for problems with 5 
plants and about 30 molding machines. J^ortunate- 
ly, as there are. several material types and .mold 
types. It can be subdivided Into smaller problems. 

Even thaugh the current moder assumes- determlnls- 
-clc production days» In reality, some orders have 
a range of production days such as either 7 -01^8 
days. Thl^ allows" more flexibility In the sche- 
duling phase, and may further reduce the' number 
of setups. The model has been expanded to, in- 
corporate variable productlofi days. (This is 
examined at each step 'in the enumeration and thus 
the schedules may nof be optimal- in a global^ ' 
sense. } 

In general, algorithm trys at most^n: combina- 
tions, where n is the ntimber. of orders*. Howeve^ 
lb computfatlons, implicit enumeration' substantl- ^ 
ally reduced the numbeyyf comblnatlonV tJiat mii^t 
be exaznined. Ju^^lng from the fact that a fairly 
good, schedule is otten obtained early in the enu- 
^meratlve process and also that a substantial 
amount of computations^ is usually spent to show 
that the current biest solution is. In- fact, op- 
timal, heuristics which will curtail the compu- 
tations are now being considered.- For example, 
we can restart the algorithm by changing the 
earlier part of the schedule drastically. when 
the computational progress slows, and eventually 
terminate all computations. 



7. CQNCLUSION . 

The' problem of finding an optimal schedule which 
minimizes the number of setups in a plastic 
molding operation is .formulated and solved by 
search enumeration. Even though the size of the 
problems tested so far are relatively small (at 
most 15 orders) , the brigihal problem combine;! 
with all technological, logistical, and other* 
constraints yields several smaller problems. 
Each of these with a clever. heuristic rule, 
that Generates a good, if not optimal, schedule 
withffi a reasonable aioount of computer time, 
should be solved quickly using the algorithm. 
It is currently in -an extensive testing stage 
for use as a subroutine - ^ : -oduction sche- 
^ duling and Inventory control v-:iiputer system. 
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AN IMS -GAMMA. S 
DATABASE EDITOR 



E. Bj^ B runner 
Gulf Oil Co rporatibn 



INTRODUCTION 

Ab peMonnel from all levels of today's, 
industries are becoming more involved with the 
**man- computer dialogue^', we see<he ^ti'iUftlMon 
occurring that more non-computer trained people 
are ;utiliz^ing its' facilities. For many years now, 
the shift has been away from having highly 
trained computer people provide^ the services to 
execute^a pjrogrim'. Increasingly, the developed 
program is turned over to the user to allow data 
tabuJation and execution to be done by him. This, 
in turn, has caused a need for ^n easy method for 
the user to do this data tabulation and program ^ 
exccUfc^oh. i. * . ^ , .v. , ' > 

^! Orie fejature that can help to accornplisb 
. tjiis Is the use of a datbase/ A. database being 
defined as »'a collection of interrelated data s.tored 
together with controlled redundancy to. serve one . * 
^ or .;niore applications^-in an optimal fashion; the 
data are stored so that they are independent of ■• 
programs which use the data; a common and con - 
tirplled approach is used in adding new data and 
modifying and retrieving existing data within the ^ " 
<r^ta baae. "I - _v 

Utiliringthis concept with its "common and~^ 
controlled approach "to add and modify- data" we 
can devise programs by which a user can tabulate 
data and execute his program easily. ' ' 

Furthermore/ this man-mach'ine interface 
- should fulfill the following objectives:- ' 

1. Meet the needs of the user. 

2. Understandability of results. 

3. R eliability. of execution. . 
.4, Ease of use. 

5- Timely: results . 

' ' <."". " '. 

EVOLUTION OF A PROJECT 

\ . ^ ^.^\y "^"^ J 973? a request. wa,s made by the - 
Gulf Oil Trading Company for 'an analytical tool 
'(i. I--P. model) fhat would determine the' 
relative value of crude oils for different reHnery 
-situations. The moderwas developed in card deck 
form using the matrix generator- report writer V 
p?:o^ct of Bonnei- Jt Moor« Software Systems, 



GAMMA 3. The model preformed verv 
satisfactorily and gave clearly understood results. 
The familiar complaint then came from the cus- 
,tomer that the mechanics of tabulating the infor- 
. mation on kcyptinch forms» waiting for and 
^ checking the keypunching and then aubmitting the' 
deck was too time consuming and prohibited them 
from obtaining the needed results .within a half-'- 
, day or less. WTiat could we do abo,ut this?. The - 
solution to ^he.probie,m see'nried to be some forn^ 
' interactive setu^'to, climinafei tfie need for the 
card.^fcck* Thta requirement turned out to'be t^o- 
fold/;a method or pro/fram to create and matntain 
an input database and then a way of submitting a 
job iptrea,rn. ' For their purposes we had nret"^ 
items numbered 1, 2 and 3 of our obfectivcw. list; 
but as yet, had not addressed items 4 and 5, 

Being that'.IBM's Iziformation Management 
System (IMS) was already resident at Gulf and 
provided the needed database capabilities via a ^ r 
Cathode 9a,y Tube or batch program,- and that th^ 
project sponsor's analyst's were familiar with 
using the CRT for financial work,. it was decided 
to use IMS to meet these requirements. « 

THF DATABASE EDITOR ^ 

The database and its programs were to 
handle. data in either the table or list format ' 
utilized in GAMMA 3, and in our design each table 
and list was to bci a database record. Figure 1 is 
an example of a GAMMA 3 table and list in card 
image form. The table in database record - * 
structure'^vould appear as in Figure 2. 

Further, since it was projected that many 
models would probably use this database,, a two- 
character prefix was defined to be used as a means 
or distinguishing unique sets of tables and lists. 
Each table/list name would be prefixed by these 
two characters to reserve it for a particular user. 
Additionally, password security was to be made 
available for each uier-chosen preHx. The IMS 
non ^conversational program to manipulate these 
records was writte n PLl and called GAMED. - 

^Martm, James, ''Principles of Data-base 
'Management^" Prentice Hall Inc.. ? / ' 
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P'Tewntly, it iss running ori an IBM 3.70/158*;inder 
VS and is acces-sible through a 327?) "CET or ' 
equivalent." • . \ ^ . 



As displayed on the next two figures ( 3;..and , ,the data. 



Figures^ 9 and-'lOv Note that this is a'^atabasjs 
display of the GAMMA 3'table>listed previously 
and Figure 9 is the text 'portion and Figure 10 ig. 



•4), GAMED provides capabilities to display and 
edit database tabled' and'lists. For display pur- 
. poses the user can show:-' ' 



1. A. table with its text 'and/or data. 

2* A list or list with text. - ' ^ 

.3* A table or list row-nannes. 

'4. .Tabl« <;olumn names. ^ : , 

5. A tabulation of tables and lists 
-in a database' set. 



7 



For editing the user can:- 
1. ]Vto.dify 



A* Table /.list t^xt • 
B* Tal?le data i 

C. Row/column .maximunn counts' 



2. Add • . . , 

.f 

A, Complete tablesVlists 

B. Table* rows*' and /or columns 
Cl List rows . 

3. Delete . ■ ^ . • * . 

A; Complete tables /lists 

B*- Table rows an(5 /or columns 

C* List rows 

ID*. Entire database set 
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4. Duplicate 



A. M^e^l^dual tables /llsts^- 

B. Entire database set 



5* Replace Row and Calumn Names 

Since the structure of aj:able/list seems tp 
have the two^distlnct parts .of text *and data^, displays 
are made with the table /list text formatted on a 
screen. in a separate manner -from table- data.- . ■ 
(Figures 5 and S.) " ^ > 

Upon. initialization, of the transaction GAMED^ 
the iser obtains the display in Figure- 7. As 
explained in the directions* *the user's two- 
' character prefix is to be placed within the ' 
parentheses,^ the table/list name that is* to be acted . 
upon, replaces the word NAME, the field with the 
word TEXT specifies the current^format in use, 
^he three' letter designate foT'^wing the- character 
"string 'FUNG:' is the function field (SEE)f the. ; 
space between 'the words SEE ^nd PAGE is the . 
options .fields, and the PAGE clause on the rigl?t 
O indicators for multiple page mess3.ges'. 

If wfe use the prefix (EB), table name 
§SvIPLEv" function SEE it;Figure 8f and depress the , 



* If we would like to modify so me* =data^ntry v 
on Figure 10, the function is changied to MOD, .arid, 
we move the. cursor down-to the^alue' to :b.e ; y. . 
modified and change it. In thi5 'case, it, is the • 
value at the intersection of row R2 and column*"'C3-' . 
(Figure- 11), Then depress the ENTER and ^ 
receivjK^^the response in Figure-lZ. . . 

-Once GAMED wais completed^ there .stiH. 
remained the problem of having the GAMMA S 
model generator access these database tables and. 
lists. „ This was accomplished.by two additional .• 
programs, called .GAMLOAD and .GAMUNLD." 
GAMLOAD will take a specially formatted file of ^ 

■ tables and lists created; by- QAMI^/LA and load them' 
, onto the database under a specified prefix, 

whereas GAM UNLD will unload all or selected 

■ tables of one or more prefixes and create this 

* specially formatted file to be readily useable by 
GAMMA,' "In^addition, another utility program 
exists to list and /or punc^ this file in card image 
form. ; ' 

After these programs were completed, the 
first part of the original two-fold user problem 
was solved: now programs existed to maintain the 
input database and make it available to the model. 
This eliminated any need for data cards since the 
analyst running the model could enter the data via 
the CRT. ■ / '.' ■ ^ , 

. ^ While the effort to produce GAMED was^ 
taking place, a second IMS program called 
JQBEDIT was developed. Its ^purpose was to 
maintain and edit a database containing jobstream 
card images. The^^^itabase^ called JOB FILE, 
was separated into members designated by an . 
i^ght-chatacter name. As an added feature, any 
w these members t^at began- with ai'job card 
could be ^submitte^ as a, job stream with a RUN 
corcunan,dv- (Example,. Figure 13* > This, of 
. "cburse, sati^ied our. second user requirement^;' 

The customer was then provided with a 
member on JOB FILE that conta^ined a lob stream 
to execute the modei. After the data tables and 
lists were updated for the current problem, the 
us^r would RUN his .job- stream and get. results . 
within the desired time^ frsume of a half^day o^.leiss,* 
' We now have 17 production L. P, models 
^nd three non-L^ P. .'model applications being run ' . 
in this manner by users at various geographical ' 
locations throughout the country* This tvpe'of 
'operation has been so successful that wc/now " 
receive requests from our user companieis that ^ 
stipulate that their models must be deveLpped to be 
run from an IMS^terminal. . ' 

Finally, I would like to list some other IMS 
capabilities that we, have added to our operations/ 
As part of a r;in, the user can optionally: 

' - -■ 

■' ■ - i; Receive reports at "a CRT and have 



2. Receive report i-as bard Qopy at a 

local printer* . ' 
. 3. Load onto the (iatabase» tables/ 
.lists created during execoition. , 



' Figure 14' shows a diagram of the overall 
operation from the interaction of GAMED to the un- 
load optimize and reload cycle, thereby given what-, 
we "feel is Si" complete and unified system, . * . 
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TABLE SRMPLE /T=10 >R=2 vG^^ 
* t ■ - CI 

. Rl RED:. - .1.0 
" R2 BLUE . 4. 0 

R2 ORRNGE 7. 0 " 

LIST <COLORS> ^T=12 

RED . •'RRIMflRV ' 
- BLUE 'PRIMflRV / . 
GRflV -fiCHROMRTIC 
BROWM --SECONDflRV 
V BLACK ' RCHROWIC ' 
• GREEN rSECONDRRV ^ ■ 
• ORflNGE'"SECONDflRV 



C2 
± 0 
"5. 0 
8. 0 



C2 

5. 0 ' 

6. 0 
3.0 



TRN 



-'SECONDRRV 



I'--:; , 
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DISPtflV CflPflBILlI'lES 

ITEXT 
• . B. LIST 



. a TABLE ^^^^ LIST RON NAMES , , ; 

D/TflBLE COLUMN teS , ' • ' . 

■■ ' ■ ■ . . ■ ■■■■■ -'^ .■ 

"E. tables: and Lisrs in a ['ATABase set 



t * ■ , . , • ' . * 

, » ; .■ .■ ■ ' '", ■ • . 

edit GAPfiBILr'-iS; 
A. MODIFV 
. ■ 1 TEXT • ' .. 
2. DATA 



' B. FtoD '. ■ - 

1 COMPLETE TABLESXLiSTS . - , ■ , ■ 
2; TABLE ROWS AND/OR COLUMNS 
3. LIST ROUS ' , 



C. DELETE , ^ 

i COMPLETE- TABLES/LISTS 
• 2. TABLE ROWS AND/OR COLUMNS 
'1 LIST ROWS . ■ ■ 

' . ■ 4. DATABASE SET . - . • 



, • D. DUPLICATE:; : 

l^NDiVIDUAL iflSLES/LISTS 
\ V ' 2. COMP&TE DATABA5E;SET' 

: ^/REPLACE 1?gw 'AN^ NAME 



, ■ Figir£4 



ERIC 



■ \ 




Tejrt Format 
Figure 5 
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GRMED^iC: )NfiME; TEXT FUNC; SEE ^^.^^ • ;/ : " 

v.: .. REPLACE Nftf'E BV SPECIFiC, TfJBLE Nflf€. l-ilTH GPTIONRL'PREFiX '( -> 
/ CHfiNGE FUNCTION IF MOD/ flDD.. DEL. 

ENTER DISFLflVV OPTIONS BETWEEN SEE RND PRGE: • 

.ROH/COL NAMES. ROH- TEXL Pflm ^ " . ^ • . - - , ^ 
• ' ■ LIST: INDICftTES SPECIFIC ROM/COL SUBLET FOLLOWS. ^ NOTE ^;HfiT 
■ FORMAT MODE must; BE DATA. ENTER $DATA AS TABLE NAME / 
. V. : :.. TABLES: LIST; OF ALL TABL^-NAMES WiTH PREFIX X ). • I ^ 
- DEFAULT' IS ROW. TEXT AND DATA .FOR FULL TABLE WED. ' ' ' 
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Figure'? 



GAMED <EB)SAMPLE TEXT FUNC: SEE . V PAGE i'OF -i 

.. REPLACE NAME BV SPECIFIC TABLE NAME. WITH OPTIONAL PREFIX <).^ 
. CHANGE- FUNCTION IF JOTHER THAN SEE EX. HOD.. ADD; DEL. ' 

■ - ENTER- DI^PLAV OPTIONS BETWEEN SEE AND PAGE: 

• V*..'ROW/Cd^ ^ ; ' ■■ . 

/ . , LIST: INDICATES SPECIFIC ROW/COL SUBSET FOLLOWS. NOTE THAT 

■ FORMAT MODE MUST BE DATA. ENTER $DATA AS TABLE NAME. ' 
■ • , < ■: TABLES: LIST OF' ALL TABLE NAMES WITfl PREFIX ( ). 

, • - : DEFAULT, IS ROW- TEXT AND DATA FOR FULL TABLE NAMED. 



'Figure 8 . [■ j 70 



' GflMEb' (EB>W^^^ PAGE 1 OF ) 2 =+1 

':R2. . ■•blue: ■ ' 



• • •. •• ■•Figure' 9 ~ 



<€E>SftNPLE .^v : DATA ™^ '^'V: f ^ V' .PAGE 2 OF. 2 =+1. 

■Ri v. : ^ 1 8000, . . zeeee. : ^3:0000 ' ," v . ; ' ' 

R2 ; ^ • 5.00,00.r v" :!^^^^^^^ . , 



"Figure 10- 
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GAMED (EB)Sl=iMPLE DflTfl FUNC; MOD , ' ' . PflGE ' 2 OF 2 ' . =+1 

ROW:/COL: • Ci. : , ■t'2 • ■ ' • C2 . - • • 

Rl. . .10000 2.0000 1 0000 ' 

R2.- ■ .4.0000. . , 5.9000 10.0. v' ' ■ ' . 

R2 *• . . •?:0000' • '8.0000 ■ • ii.0000' ■. ^ /' ■.' .. ' 



■ ^ 



GflMED. <EB>SflMPLE TEJ^T, FUNC: MOD / . pfc 'l OF^' 1 =4 

COLUMN.: 02 " • .. R0|.j:.'R2 ' VftL; ^ 6.V00 CHRNGED TO-; . lf\ m . 
FiyiSHEp WITH fiLL DftTfl CHRNGES. , ^ ' " ■ ^" 



■ Figure 12 " ' ' ' " ■ ■ 

T^, — T- — ■■ ■ 
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y/MSPGMEBB JOB 106010906004002.. ^BRUNNER X254^, MSGLE^/£L=1 
//'★miN ORG=RM070,CLflSS=SvV ■ ' .. 

// EXED PGM=IEHPROGM ^ - • 

tjVSYSPRINT DD SYSOUT,=fl ^ ; . ; • ' 

//DDI DD . UNI T=DSi<lL •VOL=SER=GSVS40. DISP=SHR " 

, SCRflTCH VOL=D5Kll=GSVS40^DSNRME=MSpa Gflr€^:^P^^^ • 
• SeRflTCH; VdL=DSKll=GSVS40.. DSNfl^1E=^^^ PREFIXES ; 

SCRflTCH ■ VOL=DSKli=GSVS40. DSNflME=MSPG. GAMED. PREF I XS' ■ ' 
SCl5.fltCH VOL=DSKll=GSVS40;DSNflME=MSPG. GAMED. PrIf 

REQUESTED MEMBER DisPLflVED=MSHA*PGM ' 
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• ^ ' " * Abstract . ^ . . 

• . Mathematical prograJiiiKlge^ is/ increasingly beiing 
combined With other matheioatic^ modeling techni- 
ques. A combination'Which has proved particularly 

fruitful in the field of energy modeling ,is that of 
an econometric model and .a linear program. Some 
practical e)cperience' has shown that although such 
coinbinpd models; give rise to unexpectedly few 

. mathematical /difficulties during solution, the com- 
puter software which is currently avail afefe is very 
inconvenient for" imp] emen'bJE»g theise models TKis • 
paper* analyses the sources jof these difficulties, 

. and describes some new^ software tools whicFuare 
being developed to 'aid iij the construction -of conr- ^ 
bfned modeling systems.- ^ ^ ■ 

' ■ • ' " .' • 

1 . Introduction - The Stat% -of ^the Unfon ' • 

Recently, mathematical models which hajye as 
major components both an econometric 'part and a . 
linear progfanming: (LP) part have come inta:vogue. 

'Hqgan tl] has described hqw such. a -model^was con- 
structed and solved at the-^^ederal Energy MfiThis-/ 
tration for the Project Independence Evajlu^ion ' 
System. Jorgenson- [2] has proposed. a caroination 
of the Brookhaven Energy LP model and the DRl ' 
interindustry econometric model:> Shapiro. [?3 
describes these and similar models, 'and discusses 
mathematical approaches to their solution. Cur- . 
ren^ly such sol u;tion {)jettiods require iteration . v 

- between vthe^ecpnome trie, model and the LP; the • 

> ecp^ometr tQ :^rojae pr^^ supply "and demand data ; ^ 
an3 - the LP ;*etums'an optimal distriibtition/bf 
comnodities to meet the. demand and. the associated - 

.shadow prit^s. The. process terminates "When an 
equilibrium set of prices is attained. This paper 
addresses, the question t>f what softjware . tools ' ' 
are needed by a- person yfiq Wishes to develop and 

: implement an. algorithm! of thi^-S-^i^ature with the/-. .. 
mtniinum' computer programming; effort. / ' ' 

'\>rt Wili 'Ve useful at t?ffe dutset-to'distin-^ " 
'guish between two separa^^ but connected aspects ' 
of^ the prpblen of combirwwig ^LP and econometrics on . 
the- computer. , One aspect Vfr that of data base . 
.management; that is, the maters hailing of the 



^his "report ;has hot undergone the^ review "accorded 
afficiai NBER publications; in particular, it has 
"nbt.>yet'been'^submttted.,fo^^ approval byrthe Epard r 
of Directors. ' v - ■ . 



original raw data into a form suitable ;fpr computa- 
" tion, and the production gf^aningful output. The . 
other is^ 'that of sj^stem integratjxin; 'that is^ the ' 
- provision of mechanisms, to control the executipr\''of ; 
the processes, and to facilitate transfer of data • 
across boundaries^ between "systems. This* paper is 
"principally concerned with system integration; data 
base management, LP^ mod^ -generation,^ and report 
writing. form, the principal thanes -of companion 
papers [5] , and so these "topics Vil T not be treated 
^ in djstail here,- * - , . 

. Let lis, first (examine the. problems that confront 
^a modeler: who tries- to use existing software.; . 

^ Kdgan [^] has described- the difficulties that arose 
during the implementation of .the'PIES model'.Cl]. . 
Despite there being* no rigorous theoretical f ounda* . 
tion. to the algorithm, its convergence was found to- 
be Onexpectedly raftid;(6 to 10 iterations being^^^ " 
requii^ed);. .Hogah was fortunate in that the.inte- . 
'grating p'ortion of the 'model could, te readily coded; 

^ iTi FORTRAN, and that .APEX, the'LP optimizer he- usea,- 
has the^'iTiost convenient interface with FORTRAN of 
any.commercially available. large-scale LP code, He 

" has stated [4]^ that the major difficulty in the 
[Jroject. was. that of matrix generation, but it Would 
probably be more accurate to say that.the reaV> 
problems' were in data*base management, as will be 
clear'from a full account- of. the' project ,[5]. Never- 
theless; although the^omfcffria tion of-tlie two "^ddels 

"/did ndt.'cbnstftUt^ the'greate'st difficulty in the 
project," it is cleai-vthat the. integration proces^s ; 

' was tiyLino.means'Sim^ple' and '.straightforward.' ' ' 

^ * ' • • ■ • . ' ■ '. ^' 

The^paucity *of published results on-TJie success 
Tul combinat>on of modeling techniques tends to ' 
corroborate 'the view that implementation of such • 
, ^icombined models Lis 'difficult in practice. . .Apart 

from Hqoan's wor)c,,^we are foT^ced to draw on our*own 
^experiehce at'NBfR-'for^ illustration of t^^ 

cultiesr which 'are.: encountered/ The -mil jor. stumble 
^ .'bliocR; is that the econometric system and" the ;LP , 
•system, having grown up separately, do not- have an 
easy way to exchangelldata; furthennare,' each hasots 
own execution. control mechanisms a'np neither wi-lT 
allow itself* to' be subse.rvient ta tne other* As-an 
example,, consider a, p.ilot implementation pf-'the " • 
. Yeast absolute residcals*. algorithm for regfes.sidn, 
in which the da'^a were set "up and manipulated -iri\ 
' TROLL [7}. and the minimization performed using the ' 
SESAMEv-LP system [8]; In view of ^fhe experimental - - 
>iiature of t^ie* project , : it was "decided to "use an V -" 
'^fecisting- LP sy^em'-ii^^ther- thcfR-toTcdde^^ 



.scratch, even- though there are modified versions of 
the simplex algorithm known which' will solve this 
particular problem more efficiently than a S;tandard 
simplex xode. Since TROLL and SESAME must each 

.reside in .a different virtual machine {because TROLL 
contains its own virtual machine supervisor), the 
only means for passijig d^ta between them was as . 
spooled card-image files. In addiction; the user had . 
to control the whole process manually from the con- 
sole because, therei was no way to pass control 
between the two. virtual machines.. This latter 
difficulty could have been overcome, at ^the cost of 
considera'ble system programming effort, by arrang- 
ing that each virtual machine could suspend. Itself 
and activate the other at appropriate times.' In 

,the end,. this project was abandoned -because the 
effort involved seemed too great for the expected . 
benefits, TROLL and SESAME were both designed to^ , 

•be flexible systems for experimental rather than 
production use, and the problem .of ^combining their 
fac|l-ities' was not expected to be so difficult. 

■Lest it be thought that NBER's/^ systems are the 
only offenders, consider the way iBiM* approached a 
similar problOT, that of combining APL with MPSX [9]. 
Again, data are ^communicated by means of card-image 
■ files." So is prog^ram control information — ' the 
APL program-constructs a card file of MPSX control 
language -comnjands and passes this file to- the MPSX . 
compiler. Th'is. mechanism' is both clumsy and . 
inefficient, but is necessary because flPSX accepts 
input of control information only in the specific^, 
form of card images',: . . 

.'** ". 'the APL/MPSX. combination, though achieved by a 
roundabout means. and with no little programming 
;effortr, was successful enough to warrant publi'cati on. - 
^ HSfuraHy* failed attempts to aclapt ah "existing LP 
system^.to a. new enviroment do not reach the' litera- 
ture,; but the author is aware from personal con.-^ '-'^ 
tacts with people in the modeling world that many 
such attempts have^been envisioned . Because of the 
obvious difficult.ies ^hictj would ensue, few such ^ ^ 
attempts proceed beyomLth^ speculative stage. ^ 

. We'may conclude;^ then, that the^state of the 
anion j's not^-good, ^/ \" . >^ 

^ ' -2.. Limitations* of Current Software 



.. Let us su]TmarizeJ:he main points arising from 
the general overview given in Secti<in 1. The areas 
, in. which current .s;oftwarfe exhibits* deficiences are 
as follow^t - • ' 



(a) Data base management and genera-' 
tion of LP'-mod^Js fjrpm raw data 



'J 



(b) Control of LP. procedures by other 



programs 

. Xc} Transfer of. data between LP proce- 
* . ijdures and other programs ■ . 

Category (a) is treated in companion, papers; [5] and' 
^ ; will -not be discussed here. Categories *(b) and (c) '* 
over! aR. slightly (for example, control* information ^ 
might'^ regarded as data passed to the LP prograrrOr 
but they. are essentially separate.. . . ^ 

iA'major^drawback of current large-scale LP.- 
_ so.lvtng- programs-, (with the -partial exception of 
'APEX) is.^that they- eannot/betinyoiced, from anpthec 



prtgram written in a standard language such as^ 
• FORTRAN <)r PL/I by a subroutine call or similar . 
'echanism. APEX allows its constituent routines tfi 
be called from a FORTRAN program once the general 
APEX environment. has been established/ Other LP. . 
systems can be invoked only by means of a special 
. language .peculiar to each system^ One reason for 
this may be ^that most -operating, systems dq^not pro- 

- vide a suiJtalple dynamic Toadirfg ..capability; LP"codes 
are nowr-so voluminous that they are too big to 
include in a load module with other , programs' of - 
comparable size.- - However, the main reason appears 
t'o be that LP programs are deemed t6\ require such a 
specialized environment for efficient operation \ 
that only, the LP /system itself can be entrusted 

^with creating this environment, 'and that tto^efore 
.thre LP system must be in overall 'control or^the ' 
whole process'. "Some progress seems to have been 
made in modifying commercial large-scale LP systems 

;to provide a more cqnveaient interface for the 
^experimental user; IBM' claims that MPSX/370 has the 
required flexibflity- [10], HoweveV, it is clear 
-that the only way to. achieve a really clean control 
interface is to design the LP "code with that require- 
ment in mind at the outset, rather than to try to 
adapt an "existing code. This may entail the sacri- 
fice of some. raw computational efficiency in the 
ilriterests of greater flexibility *6f application. . 

Transfer of problem specification data j'nto LP* 
systems is still essentially at the BCD stage. 
Seal e has championed a "card image" data interface ^ 
on. the grounds that 5uch a medium .provides the maxf-; 

^ mum" standardization and flexibility [Tl]. Whilst 
it is true that such an interface is desirable to- 

:. facilitate transfer of data between computers of 
-different manufacturers, card images are a poor 
means of>.passing datar between programs running on^ 

■ the same machine. However, Beale's philosophy has - 
prevailed thus far, and many current matrix genera- 
tor programs', such as GAMMA [12] 'and- MaGen [13], 
produce as their output a specification of the -LP 

Sroblem in a-card image format, usually that^pf 
PS/360, which has become a sort of informal standard 
Such a mode of operation doek ;have the advantage . 
that, a single matrix get^eratbr can produce output 
which is suitable for input to a variety of differ- 
ent LP systems. But the intermediate data form was 
originally designed for human readability and does 
not'lend itself readily tg; manipulation by the com- 
puter, ' Indeed, some -of the complexity and ineffi- • 
ciencyof matrix generators can be attributed to-' 
exactly this cause. 

. "Some /matrix generators, notably DATAFORM [14] 

- and DATAP^T [8], produce a. specification of the ^ 
problem /flirectl^.on a special file, referred to 
variously as". the ."problem file" or "models file"* 
This allows the LP'to. dispense with its Il^PUT or. 
CONVERT routine which it would normally use to con- 
vert the card deck into an internal representation. 
This representation also facilitates a»direct revi- 

. sion of the model , which would otherwise have to be 
performed by altering th^e matrix^ generator program,, 
yor its ,data,-;or 5oth,.-and executing the matrix 
generator again.. -The data- on the problem file are, 
however-, accessible to the user only via special. . 
routines which themselves depend on the environment 
-set up by the LP program. 



In a similar. way^ output of soliVtion Ijiforma- 
. tion from typical' LP systems is at the BCD level. 
In orcT^ to be read by another program, such data 
has to oe stored. on a file in a- format which is 
compatible with the other program* and unit records 
of 80 characters or so are ^common choice. Some. 
LP systems allow the option of filirig the solution 
data in some , internal 'form on a "resjilts file" and 
provide, system routines by which the^user may * . 
access 1:he data. This is neater, but has tfie draw- 
back that' these routines cannot be used outside/ 
of the control 'scope of the' llP program. . * 

Current LP programs fail to' distinguish 
sharply enough tetween two sorts of ^ data about the 
LP model . that pertaining to the structure of the ^ 
mod^l (its size and the. pattern of non-zero ele-»- 
menPte), and the actual values of the coefficients. 
As explained >in Section 1» algorithms for solving:: 
comb^rved models typically'' use the , results of. one o 
LP run to revise the input data for the next. In" 
such revision, only the coefficients are changed . . 
'and not ]the structure'of the model. Hence, it is 
advantageous to keep these data entities separate, 
and to make, the coefficient values easily accessible 

/ ta the "caller of \the LP code. Some modern' LP codes 
have made ^parit^J^ pro vis ion for this 'in the shape 
of '*incJ/rect"'h5oefficients — th'at is, coefficients 
whose lvalues are initially specif ied- as character* 

• string? names which are .later bound to numerical 
values. However, -the mechanisms provided for alter- 
ing the values associated with particular names. 
haye not been especially convenient.^^ . 

> ^ In a>l of 'the foregoing, it is clear that the _ 
greatest hindrance to cdmbining a modern . Ta.rge- 
scale 'LP system w1tn an econometric system of com- 
paraW&tomplexity is the irfs'istence of ^he^ LP 
, systeniPlh setting up its own environment and con- 
'lirdl meT^anisms. ^In parti.cclar, operations 'on the 
- data/base usually only po^ssible within the^^ ^ 

environment set tip by the LPvor^atrix generator, 
The jefxternaJ interface of a tj^ical "modern, large . 
Sc^e LP system has been designed to be convenient 
tpr:a1^ijmam such an interface is^ not well adapted 
f^or use by a computer' program. 'bATAFORM osten- ■ 
\ sibly has the capability of solving non-linear prob- 
lems b^ an iterative a.lgorithm involving^ dynamic 
; revistdn and. solution of successive linear pro- 
grams. However, MPS III must remain in overall 
' .r control of the process and although' it ,is possible 
' to invoke Vrom DATAFORM subr:t)ut7*nes written in, for' ' 
|fc)cample,. f^DRTRAN,. it wouljl not be possible to invoke 
another computer system ,- such as a simulator. .. \ 

3. New Methodology . ^ 

Interactive Computinc; ^ / 

We make the assumption, ^ considering new 
approaches-, to dealing with the problems outlined,. 
^bove,\that. computers will be used, increasingly in, 
^n interactive. mode rather than in a batch mode,' ./ 
' esi^ecial lj^, during, the -development phase 'of a pro- 
.grammipg project., AltFFough interactive operation" . 
a .has* -lit tie effect; in principle* on the computer 
systems required for successfully' combining LP. with * 
otheV models, it greatly affects the flavor of otir - > 
approach^" It is assumed here that the riaader, has. 
. hacl -som^-^posure to interactive compating, and , 
appreciates, both the advantages it affords the user 
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and the difficulties it entails -for the system , 
designer. ' " 

■ - .. ' ... 

NBER Support 'Systems • ' , ■ ' ' ' 

The Computer^Research Center of NBER has 
, implemented and is extending a set. of system^ " 

^ facilities designed, among other things, to aid-"-'-^ 
the developer of new modeling s^tems. ^he evolu-' 
tion^of the philosophy behind these systems ,h^->: 
been explained previously £15]. The Applications,. 
Control System tACOS) provides a shared hierarchi^eal 
file system, a.. supervisor, an^inpxjt^obtput manager 
for I/O not connected with the file system, and the*-' 
Applications Control .Language (ACOL). 'ACOL is a 

^ ^ language for writiiig. programs which .interpret com- 
mands entered from a terminal according to a' 
programmer-defined. syntax, and for controlling ther 
execution of program modules in response to such 
commands. For more detailed information on-ACOS ' 
and ACOL the appropriate manuals [16] should be 
consulted, 

, AGOS supports, as a subsy^em, U^^\ which 
provides both ^ language for specify ii^ifmathematical^. 
models arid a language in whicfiyto impl^ij^nt .algo- 
. rithms [17J.' The modeling language, like that in 
■ TROLL [7j allows equations tvbe entered symbol- 
V/ tcally, but it goes beyond TOTLL in offering 
faciili ties for- symbolic manipfllation,.such as 
differentiation. DASEL has a^ibrary procedure 
. which will solve small LP problems, and also allows 
more, powerful LP programs, to be call ed-by ''means of 
its iiiterlanguage communication feature: ACQS also 
supports XMI*, a. flexible subsystem which. solves 
.large-scale LP models. 

• Analysis, of the Model -Combination Problem 

Jn what follows, vie 'restrict our "attention,' to.. - 
one particular cpmbination of models -.-'that. • - 
ret|uired to solve the equilibrium problem Cl»3]._ 
. This vsfill make the discission clearer'and more- . • 
concrete, and th^' principles deVe loped will be * 
app^licable to almost any combination* of modeling 
techniques^ The steps in the solution of the 
equilibrium- problem are: . ' ^ 



1. 

2. 
3. 



Define the econometric model (EM). 

Define the L'P- model. • ' 

Specify an initialz/solution of prices 
for. the EM, . . 

. 4. DO while conyergerice cr-Jterion not'met: . 

a) Compute demands frort^EM, ■ 
. b) Pass demands to LP, ' ' 

c) 'Solve LP to compute opt-imal 
allQcatfon .' \ .- - - 

d) Retrieve shadow prices from LP. 

" ;> _ e) Test' for convergence;, if not ^ ^ 
^ . / , * ' convergecl,' compute fevised prices' 

' E^rtT DO r ' ■ - • 

5. Display rresults. • / . ^ ' ■ 

It is readily apparent that we require ..the EM pro- 
gram, or some program hier»archica11y superior to; 
both the EM^prograrh and the-^LP program, *to be'in • 
--corftnfl 0/ the overall execution, of . this algorithm.^ 
The; LP program acts as ^ subroutine, albeit an ' 
intricate one with, complex input and 'output. 'Let'- 
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.'Hii now cor^sider^ach of |the above steps in turn to^ _ 
see how they 'can be implemented 'with the right soff- 
' ware. ' >' ^ . v v.'^- ' ,. 

Definlt-ion of the Err . ' ' 

. We assume that the. econometric modeling laosi 
" g^jage system has a means of allowing the model to. . 

be'^specif led ^by means of. symbolic equations. 
. , ■ * • , ' ' . * ■* «■ 

, Deflhition of the LP : - 

-—TT : — - ■ ^ ...... y . • ■-. . 

^ ; ^mall LP mddets can be defined in terms of con- 
■^straint expressions "^antf an objective with only . 
minor extensions to the.EM modeling language. One * 
needs to be able to attach labels te constraints, 
so that the objective may be identified and so that 
constraints may be referred to subsequently,' and 
■ the language must permit the operators '*<=" ajid 
">=*'• in addition to Gf course, ffie LR prtJgram 

■must; be able to accept^the model in suoh a form, 
" rather. tJian -in a BCD representation; however, this • 
* requirement is easily met, since all the processing 
, to do'^wth* parsing the equations Is already present 
' in the^EM^ system. Both the DASEL library procedure 
:^nd XMP can accept an LP model in this form. • 

■ :■ ' . ■:. ^- 

Formulation of a large LP. constraint, by con- 
straint- would be .impossibly cumbersome. We have, * , 
■.tiiepefbre;; developed 'a language , called XML which . 
allows ah LP^ to be specified as constraints indexed- 
over sets, and. v*hich incorporates the I "operator 
[5].^ An. LP model m&y^be expressed in XMP in essen- 
tially the same way as it would in purely mathe- 
matical terms, with some concessions to the short- 
comings"o1rc!iDmputer typograpTiy. XML allows coeffi- 
cient values to be Specified symbolically; the 
actual numerical values may be stored in a .data. 
structure (an n-dfmensionaj -array) in the user*^s . 
file system, and a bindiqg process associates the 
XML symbolic name with the file name* of the data /"^^^ 
structure. This is somewhat analogous to the use ' -* 
of indirect coefficients in some current LP sys- < * 
tems, but it is more flexible, as will becdfiie A 
clear in the subsequent discussion of model revi- e ' 

sion. * r 

r > 



language; for example, if DASEL were used., . the. data 
could be passed simply as OASEL variables. Speci- 
fically, the demands are .DASEL variables to the: EM V 
program, which assigns values to tHf^^j.; although' 
they act as copstaats to the LP gf^-j.";- foir which.. 
they are coeffiti^nts in thfe^ rijg.|*L-hfinc- . "*de (or . ^ 
bound values). The.lTP returns ; :dov. prices 

(TT-vector) as an explicit vector #c }^ also a / 
DASEL variable, and therefore, the -iiN^gj^am can. 
retrieve values from itV The difficulty with thisy 
approach "is that it requires , the;user to keep track 
of the elements >in these vectors by numerical index 
values; this is satisfactory for a small LP model, 
biit impractical for a large one. 

'^"If ohe\' attempts to extend thris' approach to 
,largef LP models, other problems arise thaf><keepiji5 
track of variables by index number. Even with a^ . 
computer which- provides virtual storage, the^. avail - 
•able "core" storage may be. exceteded,. because the ■ 
code ajid data for both th^ EM and';th^ L> exist" 
simultane'ously. Further, the'tP protjram constructs 
' its working data from the mbcieling-language descrip- 
tion of the LP from scratch .each time; for a,.small 
IP this is -a reasonable' approach,. but it wocrVd 
clearly be grossly inefficient to generate a large 
LP. problem each time from an XML specification .-when 
the -only changes concern' a .few right-hahd-side 
.values. Finally, when retrieving solution values 
we-wish to refer to rows and variables in terms of 
the XML- mode] specificati6n, 'arid use this ^s a . 
selection mechanism, because the full primal and - 
dual solution to a large LP.woulcl occupy u.nnefes- ji. 
sari-ly a good deal of .memory if it^were expressed. 
_ as. unpacked' vectors. — • / "^^ 

Efficient Revisieti of LP Models -' 

* ^ , '. ' . . S"'. . ^' " ■ 

In order to solve thj^prb^lem pf lack ,of space ' 
.we^ require* a supervjj sory^rjogram which passesr con-- 
■ trol alternately^to^the Efl.'prograin anS the LP pro- ' 
gram. Jnjh is ^way^t^ifDh-bf^ the data^areas and - 
execul 
. sucK^ 



2«:JUdy l^y- bverl aid , 
v€y. 



ACbL provides- just 



Specifjcation of an initial solution ^ . . ^ 

t * , ■ ■ 
The precise details of this will depend, on - 
the EM system, but any good system will.;mak(B gon- 
struction pf a vector of valuies a simple process/- 



Main Intei^t^e Loop - \ - 

, If the 1anguage-:of the EM system allows the 
"LP program-^ teVbe invoked-by subroutijrie^call or 
similar mechanism, -and if it -also has j:Q,nstructi^ 
' f oV testing ;''branching, and looping, thVs itera^ 
tive>liop^cari be controlled by the EM 5jystem.jf^.Soih 
TROLL and DASEL provide the appropriate constnaicts.,^ 
bur only^^DAStL has LPras a cali[able furrctionV' 

AWe can -assume that steps 4a*and '4c are 
achieved slrfTply by^ invoking the respective pro- 
grams, and ^hat the EM system/provides .^he. facili- ; 
ties:. for comorfting^the revised priced Ifestep 4e^ ; 

The passi-ng of. .detta in steps 4b and 4d is the- 
rmal xruxs^of the problem.; It is easily solved if i 
th^LPvts s%ll. enough to be expressible in the EM' 



te,ly,i if , we.* remove control from the 
erand' arj/eff icient mechanism for - 
Inakin^^am re ViTfcnTS^o , the LP model , we encounter 
s probJ^rSkith. the scope of the data. - This aris^ . . 
■ as foll^^^^Vo solve an JCMLirabdel, the -use^* sup-^ 
;j)l ies' -tipe.da'ta' (as 7i-dimensional arrays) , . specifies - 
. --^ a binctmg of XML identifiers to the data, and , 
.invokes :XMP .with thVneiTne^of the XML model as an 
• afigumeVrt. A^t this level 'of operation, data entities 
^.gi^suct as "right-hand-side" which are internal, to the 
j^^^.lP system are not accessible, and. so values in'them 
'M -^ cannot be changed. It wduld be possible to. allow. ' 
•^^V- access to these internal data if either they .were; ^ 
* V pr^sVrv€d«=q*n' cere, between* cal lis to the LP code 
(ACOL^ermite this), or if ttje data were saved on 
" the- file.. system,, but both; these' affproaqhes are open" 
tc obSectionv ' . . -^.^ .. - " " 

The good.fcolution to th^'problem lies in taking 
. ; advantage .of tbe hierarchical ^structure of XMP,- 
whicfi Was -designed, to facilitate. just such use. . 
T^e top-level procedure XLPSOLyE consi^ts^ of caM$ V 
to *the secorvJ-level procedures GETPROB (which^ con-' 
ver^s the problem from* XML. form td iiiternaV -form) 
and SOLVE (which performs si mpj ex i^terat ions). 
. Internal data are accessible' at th*is.Tevel, and so 



small revisions to the right-team^-side valijesi.for 
example,- are. .easy; system- r6utines are provided for 
t^his, in dpder. to maintain modularity in the Parnas 
[18] "Sense • The particular right-hand-side ele- 
ments being, changed may. be referred to by the name 
of. the- corresponding constraint expression, so that 
th^ usee need; riot. be concerned with n 
indices'*. Jhe conven^tions governing the use of 
names are explained in the \Section "Retrieval of 
Results''': ; We can then .revnrite- Step 2 of the'algo-, 
rithm on gage 3 as: * ' . 

2. a. Define the in XM. . 

b^. Invoke GETPRQB to convert to. internal 
.• . ■ • -■ ■ form.., . :\ ■ ''''^^ ., ■ 

and Step 4!feas: - • v " ' ' 



/ \ ' Invoke solve: . . * 

Observe that GETPROB and, SOLVE- can be invoked" 
independently of XLPSOLVE, something which' isnot : 
possible with current commercial LP systems. It is . 
possible In XMP because data "Entities are passed as 
arguments,, and therefore no routine is required to 
set up a special environment in which these proce- 
dures operate. \ .. 

Retrieval of LP Results 

' ■ y , 

^ . The* particular problem we are considering 
requires only the, retrieval of a few shadow prices. 
In general, one may assume that the solution^ values , 
mos.t af ten requested will be prijnaT and dual > ' ' 
values ahd that tabl^ij, values will be required 
mucffless frequently;/ hence, XMP. makes primal and. 
dual values^ especially accessible. The internal 
form of -the.final basi^ is also made accessible . 
so that it can be saved and used to initiate a 
future executjon of.:SQLyE.. , ' . . / : 

. - ' As with- revision oT a model*, ;the scopie of 
data presents a problem with retrieval of results. 
However, we cab overcome the problem by 'using a 
special, feattyre of ACOL- When .simplex iterations 
have finished, SOLVE makes a recursive call to ■ 
J\COL {recursive because ACOL inyoked SQJ-VE in the 
in the first pUce); SOLVE -i^ ^tiMT active and . ^. * 
therefore all its internal data are. pres^-ved. Jn- 
responses tV the recursive" calT-, the ACOL/ inter- 
preter starts to read from whatever environment, 
invoked SOLV*... Jn" this 'particular-case^ SOLVE 
was iTivoked. from an- ACOL program ^ and^o more of ■ ; 
'this program is read. The statements/that, are - 
read are requests to SOLVE for solution infontia- - 
tion which -are implemented by the ACOL program; ; 
rit calls en tcy Pflints in the SOLVE program which, 
return solution^ values. The solution values are , 
then passed via ACOL to the JBI program? There - 
'is an explicit ACOL requestr'to un^/ind-the recur- 
\sibn^nd leave the SOLVE environment. 

- " >When re,trieving soluti5r^information^*^he user' 
must be . able to refer. to the constraints and vari- 
ables of the LP model by meaningful, names. With 
conventional matrix gerrcrator-systems thisv'is 
conceptually easy, since, all these systems require 
that^thesuser specify these names precisely in ^the 
form of 6- or g-character concatenations. ^: Normally,* 
these' hacies will have been generated in some . .. , - 

'regular-<fashion which allows s^electior^of "the 



<lesired solution information to be achieved Tjy a- " 
masked-name matching^technique. However, XML has-, 
no concept of i names of this type; its names. refer 
to whole classes of .constraints and variables, with 
individual merfibers of a cTa'ss corresponding to 
. particular values of elements of indexing sets. 

*. : Jhe style\of naminq; adopted by XML is very' 
convenient for /;selecting.*classes of element's of the 

■ * model... In^ the particular case, with which we are 

concerned/ suppose the set, of xlemand constraints is. 

■ named* DEMAND and is indexed over regions.. of the 
country and dver^ typ§ of energy;. these index sets . 

- might, be . '^NE'^^W ,^NW" (for New England, , • 
, Middle Atlantic,;>-'l , North- West) and . • * 

-".Ei " > '*NG i . ,-"R0 ' (for e 1 ectr i c i ty * - " . r . 

natural gas , . . . ,r^esiduaL oi 1 h respectively. Then^ 
the whole class of. consT^jnts is .referred tcT by 
the name DEMAND; -cross-sect-ions would, be denoted 
by, for example, DEMAND("NE",) or D£MAND( ,."NG" ) ; 

■ and a single constraint by a name-such as'.\ 

/ DEMAND("NW","EL").\ The ACOL request ta retrieve 
^ the shadow prices for the dem^ds might be some- 
thing .like:. V • » 

/ ■ \ ?l {"pEMi«^DS"v"PRICES'*). 

where PI is the reqciest. The quotes. denote that 
each- argument is a lliteral character string; PRICES 
is the .name of a DASlEL variablie in whfch the values 
of the shadow prjices Will be returned. The values - 
are-^ returned^ in the same order .as , the members of 
" the corresponding index sets, and arethen available 
to the EM program for*, further ^computation. 

Reoorting of Final Results - . " " 

v-Since .the LP' program is essential ly-a..sub-r 
■ routine on the whole process, the EM program has 
_the primary responsibility for reporting the final ; 
results. Both TROLL and^ DASEL haye-extensi.ve 
facilities for manipulating data and displaying data 
" in both tabular form and as plots^. It. .is apparent 
however-v *hat the^Vesult retrieval operations ;dis--; 
V cussed abS>ve form a partial,set of primitives for' . 
/the construction of; an LP report writing /sys- 
tem. , ' V ' ■ . ; .1 . 

' ' - 4. Summary and Conclusions - 

A^ous^ the discussion in Section 3 is largely 
i. centered" around'a particular model .combination.-, the 
principl^^ expounded. admit of ready general izati-on,. 
" • and -the software described^is capable of dealing ^ 
. . with many other possible -cdtnbinations in a straight- 
forward rtanner. • The jnaih po.ints to notice are as 



follows: 



■ (a)' ACQS provides a uniform file system, ^.- 
^- 4^ - / . thus eliminatipg one common* source of . 
. incompatibility, and a dynamic Toafder: 

- (b) ACOL allows one to write code -(in. PL/I, 

.FORTRAN or whatever) which -may be invoked 

either' from the, terminaT" or from another 
- program without the code', having to . dis-- 

- . -tinguish between the two. Thus- the . . ; 
• . • algoi'ithmic modules of a system-such %, 

- - as XMR^andVtheir associated ACOL driver^ 
programs are readily incorporated into. ■ 
• • • systems . ■ Also , ACOL al Tows error ; , . 



' <:ond1tions to be treated in a uniform- 
manner andvrt the .appropriate Jevel^ 

(C) DASEfi^provtdes^a suitable language, in ' - 

. .which to' specffy econometric models, and 
^ " algorithmic procedures for operations ^ ^ ' 

such as .regression . and ^simulatfon. It 

also has facilities^for.tnaintainirtg a . 

data, base, man1pulatinc>Tidata, and d1s^ ■ 

playing Uaiit. . . " 

V(d) XMLis a convenient language for speci- 
V* fying large-sc*le<P model s» and XMP is 
I : ^ ^-^ a .subroutine capabl^s.o'^' .splving such 
models, \ - 

Oui* view is that extremely flexible^ applications . 
software results if carefully structured libraries, 
of procedures are available to. be connected together 

,,as 'a user desires under the general enviroiynent of 

.a suitably tailored operating system. 
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■ > * ' ABSTRACT 

3W.S paper descri'bes^ tbe fundamental 
' conoepts of data basej management and ' 
proceeds to suggest^ -the utility of 
these concepts for the data > ^Ty ll in g 
aspects of* mathematical programming* 
* A general networK-based data manage- 
ment system is xised to illustrate 
data structisre definition, data ma- 
nipulation and n^rprocedural query : 
dapahilities with respeqt to math . 
progrsmming* The emphasis is upon 
.flexibility and convenience for both 
the imp l em en t ors and users of mathe- 
matical programming, algorithms . 



r 



-Implementation of mathematical programming 
algorithms entails the storage and manipulation 
of lar^ -vdumes of data. Our principal obiective 
is to suggest vays in vhich both, the iagplementors 
and users jof mathematical programming may benefit - 
firom developments, in the growing field of data 
base management «' For iz2^)lemento(rs^9 data , base 
. management ^can provide; powerful data storage ' '~ " -.^ 
mechanisms and .f)Biclle data manipulatior 'capabil- T 
ities. The storage mechanisms are pow^^iil with, 
respect to" their capacities to handle complex data - 
relationships and both nuzheric and non-numeric 
data within a single data structure: The data 
maziipulation' capabilities allow access to -^he . 
data yithout concern for overlaying or management 
of auxiliary storage devices. :For xisers^ data - 
base maoagement can lyovide' enhanced data main- 
^tenance facilities including data editing and 
problem formulation. 

We camaoioe. with a reviev of { 
data base te^jUnology and concepts • 'g' These are 
utiJtized to illustrate specific ways in which . ^ ■ 
various issxiek-^f mathematicaJ^ programming may 
be "^i^ated with data base inanagement '.techniques. • ^, 
So, the emphasis. oh data handling, rather than ^ 
mathematical cQN^omputatlonal details,- Naturally, 
ttie^-development of an algorithm will exploit the 
the' available data structure and data >>?n«?i4y»5 ^ 
capabilities -S^'As will be noted shortly there are 
several varieties of data bases'; the networlc vtoi- 
ety,, which is the moist flexible and jgenerai;^:.^;^L 
be used for illustrative purposes. A laziguage' ±e'\ 
presented which .allows full manipulative capabili- . 
ties, yrlth respect to data organized according to * 
network structures. ^This language is ccanpletely 
independent o^ the types of data that arc stored 



and it may be used within the copfines of such 
connnonplace languages a^ rCKTRAJST, ALGOL, of CQBOL; 
An e x ample is provided demonstrating how this data* 
manip^Llation language can be used to accomplish 
the pivot operation. Finally, a high,-liBvel, Eng- 
lish-like, non-prQcedural query language is de- 
scribed. The quexy language is independent of the 
data structure and automatically interfaces appli- 
cation routines with desired data. 

DATA BASE COTTCEPTS 

A data base has two major ^attributes: its 
. logical structure and a collection of data values 
which are stored according to that structure*. The 
logical structure (also called, the schema) is ef- i 
fectively.a blueprint describing j^t types of data 
valiaes ^i^ be included in the d£rea/base and hc^^ 
each a^^'chese types is related to^the other types.' 
The schema^ sesrves as the basis for all data mani- 
pulation ;'^addit ion, deletion, retrieval, or modi- - 
fication of a data value is accomplished by refe- 
rences to its corresponding type in the logical 
structure. ^ ^ 7 ,_• 

Every data base stanicture can be described in 
terms of tl^e basic feattnres: data item types, - 
record types, and sets. A particular schema is ^ 
compose^ of data item types which are related to 
" one 'another^ by record type's and "by sets. So -data 
item -types axe considered to be the most fundamen- 
tal^ "building blocks" of a logical structure! Each 
data i^bem type is identified by a unique n«^ ] 
indicates a distlnct^ type of data. -For instance 
. - .eac^ of the 'data' item ;types'^VARIABLE-ip, VARIABLE- 
■ DESCRIITIOIT^ CONSTRAI35T-I1), CQI^^ 

C0EFET:CEEHT- VALUE, and COEFFICIEKT-SOURCE indicates 
a distinct' type of data which we may want. to in^ ^ ^ 
' dude in a data base . Each data' item type re^re^' 
sents many- occurrences of data values of "that^ type 
. within the data base. It is ijr5)artant to uiider- 
, stand that a data, itemt type is- a coorponent -of the 
schema, whereas a data value is an* actual* instance 
of dataC^of some l^pe. ■ For example^ the. data item 
type VABIAHLE-m may repres'ent "the data value oc- 
cxffrdnces 'Xl' through 'XlOO''. -.^ ^ . 

A logical structure. is fomted by relating the 
various data item types with each other. - There are 
,5 two varieties of relationships aggregation and - 
association. A record ,type is a named eiggregate ; 
of data item types. For exaisple* the record type 
VARIABLE may be ccBxpo^ed of the data item types 
.VARIABLE-ID and VARIABLE-DESCRIPTION. .Fljgure la 
gives a pictorial representation of this, where the 
record type is shown as. a. rectangle labeled VARIA- 



BT. F and enclosing the.naines of its data item types. 
Just as a record -^rpe is an aggregate of data Item 
types, an occurrence of a record* type is an aggre- 
gate' of data value occuarrences (i.e. , one data value 
occijrrence fdr each of the data item types which ■* 
constitute 'the record- type). A sample occurrence* 
of the record type VARIABLET is 'Xl' and '•AMOUNT OF- . 
RESOURCE 5 TO, BE USED.' 
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context; of linear progranning*' : If ve reverse the • - 
direction of the axrow in yiguire JLh, then- the struc- 
ture, no. longer supports tlg^ relation betveeh coeffi- 
cients and variables vhicjtt is^inherent in lln- 
ear programming'* " Thus- an'.' important issue in data.> 

-base .management is the -design of a' schema that cor- t.r 
rectly reflects the relationships ^among Item- typesy .:. 

: ^ Not only does a set , .provide' infcormation about' , 
relationships among occurrences .of its owner* and ..^ 
member record t^pes, but it ilso' permits the l aem-^ ^ , 

""ber occtirrencesf associated; with each owner occur-^5V 
rence to be ^ logically* ordered ffor purposes" of ■ -v 

■storage euid access) abcording to some criterion. / 
Tor example, , given an owner occurrence of the' sejb 
HAS, itas membffr^^ocbtarrences may be: ordered/in an 
ascending fas*hion according to the values of their 
respective COEFFICIENT-VALUE occurrences. 

Figure 2 . displays fitn example of record occiir- v 
rences organfzed^ according to the schema pt Figure , 
lb. Three : expressions are shown having five vari- ■ 
ables and a total pf twelve coefficients. In the • .:. 
diagram beneath. the expressions,' circles denote 
record o^^ccu^ences of the type shown in the right 
margin. The arrows show which member occurrences 
are owned by ^each owner occurrence, via ^the set in- m 
dicated in. the,righ^"margin. For purposes of iiia- 
grammatic clarirfcy/ record occurrence details are 
not shown; e.g. no data values of vASiABLE- DESCRIP- 
TION are included in ■ occurrences of /the VARIABLE 
record type. Also notice that -no zero coefficients 
are stored. The difiigram clearly -displays the' idea 
of A set. Each* owner record type may.be associated 
with mai^ member record occurrences, but no member ^ 
occurrence is associated with more than one owner 
occurrence. * . 



Tigvr* 1- £xA»pXc or Structurml Cc u pooant* 
of ft. £»tA 



The second kind of relationship allows 
record -types (and. therefore item types) to be 
associated with each 6ther by means of a set rela- 
tion. This notion of a set was popularized by;:the^ 
CODASYL Data Base -Task Group (DBTG) Report of ' I97I 
[I3. The DBTG notion of a set must not. be -con- n 
fused with the. concept of a mathematical set; there. 
■ is np "direct relation bistween the two. 'A set. is. ' . 
described in terms of an owner rje cord type and a'- 
member record type such that there is a one-to- 
mani' relationship between each occurrence of fhe \. 
owner type and .occurrences of the member record 
type. But although there may be .man^'- (or one or ^ ^ 
zero) member occurrences associated wi'th each 
owner occurrence, for a given set .a particular J* 
member- occurrence may be associated with no more 
than one occurrence of the owner record type. , 
Consider the record types COEFiFICIENT and VARI- ■ 
•ABLE shown in Figure lb, whezte the former is an- ' 
aggregation of such data item types as . COEFFI- 
CIENT-VALUE and COEFFICIENT-SOURCE . " The two 
reocrd types are related via the set named HAS. 
This set is depoted by an arrow pointing from- its 
owner Record type (VARIABLE) to its member record 
type (coefficient) . Recalling the definition . of a 
set, we observe that each occurrence of VARIABLE 
may .have majay occurrences of COEFFICIENT associated 
with it, but a given occurrence of COEFFICIENT may " 
be associated with no more than 'dne occurrence of , 
VSutJCABLE.^' Thus the set HAS properly describes the 
relation between variables and coefficients in th^' 



lOOCl * 50X2 - 7Q£*» - 3CQC5 • 

- lOQ ♦ 13x3 - ioc** - 9<ac5 
♦ 3XU ♦ IOCS 




- ' 'Sa*e« on .tbv Schcsa. omps* lb. ■ v 

It is readily apparen'f that -the structure -of 
Figures lb and 2 does not distinguish between co-. 
efficients of one expression and -those of another. 
In a subsequent section, we deomonstrate a manner 
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: ;-ia which thl,«| ^ short Aaing can, be remedied;* " 

;'-^3,^\jjllali2ihg::^h^ termlntld^ \iniT^^'- ' 

■ "duced in'the pwcedin€:«w^c;;tit5n, we can illustiNa-W ^ ■ 
the varieties of /data base structures which <ian 
exist. The most elementary data stj^ucture is" com-" 
posed of a single record type. This is analogous 
J to a K3RTRAir;Brray. A slightly' more complex circum- 
' stance occurs 'when data jajjaes' are orgeuiized into a 
number, of disjoint (1.(^7, no set relationships) re - 
* cord types; this, correlrocnds somewhat to'a igrptrp of 
FORTRAN arrays! FiguTe\3a shows a linear-^ructure; 
. this means that each recbrd type ' lis the owner i-or at ^ • 
..-most one set^ and. is the rtpnher ■ of no; ibfe thanj*6he 
^jgsA-, ' Figxxre lb is an exaaple/ of a. 'linear structure, 
A cfetta base may be composed of severeil disjoint lin- 
. ear structixres," A more flexible^ structrcre, the 
; tree, is depicted in Figure 3b. Observe that in 
this type of structure a>re cord type, may be the 
owner of more than one, set, but it can be the mem- 
..ber sf at most one.' Thus there is a unique path 
" . be'Uieen any twa record types. Figure 3c shows a 
neliwarX data- structure. This is the most general 
of 1^'^^ cJata structures that 5tre des cribable i*n 
terms of the three features: data item type, re- 
cord type and set. That is, the record type, lin- 
ear "structure and tree are all special cases of a 
.:'network.' ' ^- ■ ' . - 



oif-^a^pati:- contains no Reference' tp;^rect ion. The ..* 
dit-ection of arcsi^(sets J^withi<n a" iAth\pffers no^:" 

*'liHpe"difeat to-'the/.traver^^ of recoJ^; -types ^-Tijg^^ 
lare^JUa, i^ejents . a special;type or^paih^^ich may^/;^ .' 
exist , iirtt network structure ..; /The* ]^th "between 
VARIABLE and ipCFRESSION consists ofcthe two .sets 
a^S and CONTAINS j related through £he record' typ^* 
COEFFICIENT. ^- path of this scart;ns used to indi- \ 
cate a many-to-many relationship^xfetween variables 
and expressions. That'ijj; a var^ble participates • 
in many^expressions:/fuid. ari.'expr^sjlon contains -mai^'" 
variable^k^v But^ each occurrence of ^ COEFFICIENT is 
^ 'owned! ^y one "occurrence of VABIABLE (via the: set.- 

■ HAS) amd 'by' one occurrence of EXHlESSlbN {via. the. 
set' CONTAINS ) • . "It j . tliere.f ore;', relates a .vari^able 
and an expression without violating the definition 
of a set. This is exemplified ^n Figure 1+b iStsing 
the expressions of Jlgure'^S^a*^ ' • ^ ^ 

V . v.iiT.wTV « V* ■ EXPRESSION 
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Slnce-a record type in a network may be the • 
member of many sets aiid also the owner of many sets, 
multiple paths may bxatft between two record types. 
The terrfi path.indicatSa a sequence df sets which 
are. related via record types, such that any two re- 
.cord types on the path .are xelated by a unique -sub-\ 
sequence of the path's sets(i.e. , the path contains * 
no loops). It im&t be eiirphasized that although a' 
path is canposed.of. directed arcs, the definition 



LANGUAC2: FOR LATA BASE DEFINITION . ;.- " 

. We preface this, section with a few wards of 
caution*^ Xt-'Cannot be. asserted too strongly that 
the diagrams in Figures 1 through V.axe not "flow " 
• charts or' PERT diagrams;- they can in no way be con- 
^sidered to "re^esent algorithms*. As already, stated,.- 
the diagrams with rectangles depict logical struc- ; : 
tirres according to which datat may be organized and 
a diagram with' circles gives specific. examples of 
'data values whi^^ have heen or^mized on the basis 
of some schema. The diagrams of logical structures 
provide a sinmJfe mechanism for representing' and com- 
niunicating aTOut a data base* However , they . are .% 
no means ^simplistic, but rather quite powerful. . 
This power arises fipom the capacity of a logical 
structure to s upp or t -, the storage and access of . data . 
values , in such a way that a ' dieita base user need 
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•''•j^r ao^l5e expllctay' concerned vitb: the complicating 
. . ■< factors o^.jpihys^cal.stcrase ajid access « For in-r 
stance/ the particular data base management system 
■~ that, is • examined in this -paper is implemented in , 
, terms ^ of doubly lixiked(list5 vhich are stared on 
a direct access devi'cev -The physical storage ajid' 
retrieval oir data for this system requires extensive 
' ' ' pointer manipulation' and the huffering of hloclcs of* 
• data ft-om the" direct -access device • However, the ■ 
. • typical data hase user is concerned with none -of 
•this; physical aaiiipulatiohs are' automatically aCi» 
^ ^7^.. complisbed "by sutmitting' requests in terms of the 
V'^-l9Si.cal, struct lare.."-^ ..'^ /' 

•V ""i^ * . 5% loglcal'-''fftxuctt^^.<)f a data >ase .is '^formal- 
■- "^/deliaed'Ti^ a Ijata !)e'BCrl5:ti6n -lAngu^ (DDL).. / 
^ We illustrate thi^ "by" us^ng\%iiei IXDL. of a particular;' 
data management system: the GenercQized Planning ' 
System (GPLAN) developed at Purdue University* The' 
" isDL for the data structure of Figure i*a is shown., 
•in Figure 5. In this version of DDL all identifiers 
"are restricted to four characters in length- As 
' the "figure indicates,', "the DDL is non-procedural. ' ■ 
Each record type is followed by the item types which 
conpose it. Each item type is- defined in terras^ of . 
•its nasser, its type (e.g. integer, character, real) 
and its size. The - size of a, character item type" ihs 
given by its nmber of character s; other* size spec^ 
ifl cations are in terms of computer: words. The DDL' 
description of a schema serves as input to a.DDL . 
^ Analyzer which produces schema tables. These ta- 
bles are subsequently used Jby the Data Manipulation , 
Language (LML) for creating and accessing record- 
occurrences. ' ' ' 



allow :the: data:\basie; user, to .pearXora the following 
types ""of functiorjji^:!)' open.and, close ifae data" 

- base; 2)"..create ahd^delete- record '6tcuri^ .• 
seti current indicators;, k) add and remove record, 
occurrences Trom, sets; 5) store and retrieve data 
fTcm reccxrd occurrences; and 6) search through sets 
.for particular record occurrences'.' In the GPLAN 

~ implementation each 'DML cpxnnand. is a call to a . .y; 
FORTRAN subroutine. Thus the'DWL can be considered ' 
as an extension to the FORTRAK language. ^This ex- 
tended FORTRAN provides for all varieties of data ' 
sl^cture&^r^d furnishes po^rfiil^data handlix^ 

" 'capacittos, ' ' / - 

iWordtej^ to danonstrate ML commands and how ^5^. 
they ma^ be used, a detailed discussion is provided . 
of a FO^mAH subroutine which- executes a pivot op- • 
eration". This; subroutine, assume svthat- ar lAtr'ix has • 
been stored in a "data "bsuse whose jjtoical structure 
is -shown in Figure 6a. ' Figure 6V g^^^^ theP DDL of ir V 
this structu;re -,^^j^e ' must point out tii 
is allocated for any zero^ valued coei 
duririg or after, the pivot operation. " 
plays the pivot, subroutine with input arguroents f or - 
'the ■v'arlable leaving the basis (fiVAR)' and the vari- 
able entering the basis (NBVAR). 



no " storage, 
fitoient before. 
Figure 7. dis- 
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. DATA Iji||JIPIEATIC3Er LANGUAGE • - • 

• .The "iSPLAN Data Manipulation Language [2], [3^ 
consists . of apporcxizDately seventy commands which 



Injure 6. Scbea^ DDL UMtf tgr 
AI<arlti»' of 7.' 

. Hoil^ith argmnents of IWL ccanmands- refer to . ' 
data item^^OT^s* record types or ■'. sets that '.-are- de- 
fined /in ?the s chema ; aH other, arguments " are POR- 
TRAIT "5ariables. The iWSK commnnd (Find Member based 
on Sort Key) locates the member of the. set .S§ whose 
socrt Xeiy (the data item type NAMB) has tfee value 
given in the K)ETRAN-variable BVAR^ . -Each record 
occurrence in the data base is identified by its 
own unique' data bsLse .key« GKM^(Get Key from Meanbi»r): 
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'gets the .k6y,.or;tSe^incmber of. the spt SB (i.e., the." 
• membej^. vh±ch^lms',Jtist\been located by .aijd ; ^ , , " 

placcff.'that -Trtilue. in the: I^5RmAN'v^ 
■^^N^, SPM ("Set Field ^l^ased on Member) sets- the ;yal- 
* . ue , of NAMB- . in the member occurrence of "SB . to. the 
'value of the FORTRAN variable NBVAR. In these 
tlj^e c&imsf^ads, we have removed BVAR .-^Krom tie set 
oi-'^asic^VBriables, ^ind.we have e^ert&yi^XJBVAR irtto 
the basis Similarly the next three ''cotnmands re-r . ." 
. move" NBVAR* ftom the. set ;of non-basic variables a^^f^ 
•replaces itvvith BVAR; KEYB new contains the key 
\. of the fpraerly basic variable and KETN is the .key ■ 
of the formerly non-bai^ic variable, 

' Now ve find and jupdat^ the pivot element . • Tbi 
ccnsmand SOM (Set Owner of- one set based- on- -the. Mem- . 

• ber of another, set)- sets the' owner of ^the set ROWy 

• based on the' current member* of the set fi- n^ T7»» fy^ ^^- 
that'^the current member: of the set SB is the form- 
erly basic varlablev. As a! result of. the ^SOM, the 
^ , current member^pf ROW is^an occurrence <^ the record 
type CQSF. 'The next two cccmands are used to, deter- 
mine if this coefficient was a/member of the COL 
(coluinn) owned by^the formerly non-basic variable. ' 
-'S^^^^^p^ ^^^X. occurrence, of COlT based on the 
•> cur^^^'^ember^of . RCW^. ^Sijice/ a strictly' ^"^^ 

'one-?o-i;ikny^#ea^^ having a member of COL 

immediately five's us its uhlciue owner. .GKO^ts 
^he key. of this owner and returns it in the variar' 
ble KETC. We check KEYC against KETN to find^'if we 
have located itte coefficient which was " the member 
of the COL vOwned )jy the previously non-basic vari- 
able v If this is not the case, theri we f^n FNM' 
(Find Next- Member) which locates the next occtar- 
rettee of coefficient' that was in/the ROW of the 
formerly basic variable. If there is no next member 
then aji error has occurred. If a next coefficient 
of the.fonfl^jly basic row is found, then as before, - 
we determine with -which oolunm that coefficient is 
■. associated. XJpon' finding the* pivot element , .wf^ up- 
' date.lt appropriately' and retrieve- its key (i.e.., 

the address of its location in the data base)* . 
■ . ' In; order to update coefficients in the* pivot 
column', we first set the .owner of COL based on 
KEYN by iising the SOK command. • Usage of the other' 
comnands appearing JCh this procedxire already 
been illustrated' with the exception of I3RM- (De- ' 
le'te Record based on Member). As can be seen - 
from the code, if "the updated value (y) of a co- ^ 
efficient is- zero then CRM is used, IThis canmand',. 
deletes the. current- jQetaber of COL (i»e. the oc- . 
currence .of COEF whose updated value would have 
. .been zero),,. ■ .' . . 
. ■ l^pon ^>udating an .elemezit 'in the pivot, column 
' we "proceed -to update, each coefficient, in the. col- - " 
.-umn element's row* comnands for this. are much j> 

the same as those given above . There are two new 
^points Jtifire. FFM'" finds. t'!he first meiab€S^6f a spe- 
..cifi^d. set; if ^FNM returns a negitftve, value in the 
^ISEIR variable^, then there are^no more members in the 
set", bedng considered* As the code ..indicates,.' for; . r - 
"each non-zero coefficient in the pivot row, we. up- / - 
■date the corresponding coeftt>S.cient in the pivot col-. 
^ umn. element ' s " row . . Now if there • is- no such* corre- 

• sponding." coefficient,* (ire., the coefficienf was 
zero)' then, one must be /created and given the proper 
value 'The command CR creates a record occurrence 
of the type specified in its first arguroeiit. So" 
statement'^ 60 allocates storage for a record occur- 
rence ofrthe>^ype COEF; KEYNB is'reti^ from CR 
coritainicg the uniq.ue data base key for the created 
record occurrence. The niaxt coaimand' (Set 



, 'Field of Record)," inserts the AraLue of j(^\lnto^ 
: the VAir^tfeld {i.e. , data :item type> of ihe^ jx»st • . ' 
created record occurrence of COEF, Finially^tbis. 
record occurrence must be associated with the. pro- 
' per row and. coluinn .t&rough^ the sets ROW and eolT. 
This iy accgraplished by AMS (Ad^. lA^^r to Set)-.. " 
^Fgr tnstajice, ^the final AMS shown a5Se the- record. > 
^ .occtirrence - of CO^^to the ap p ropriate owner occxar-, 
2:enc« hi the set ^OL. 

■ "If there, is no need to create an occurrence 
COEF," then the existi:^ tableau element is updatet 
"iin a straightforward manner. • If- .-Me. ypdated, Valiie 
is zero^hen -the record. occuriVno/^^of ' COEF .is .c^ 
ted; the Vpace which it. has occupied is retu^teed to 
• a pool of available ^pace fcic future uses- ofrae CR" 
..: command. Finally aftef^coefficients of "the row [-^ 
associated with eaph^pivot colimm element ""-(except 
for the .pivot element .itself J arte updated, the pi-*. 
vc5t row coefficient's are properl^ adjusted. . ' 



The purpose of \ the forego: 
be suggestive of the\nature ' and 
• , DML. A ccDiprehensive deeciMptiaa 
^ . iXicluding^physical.ibrplement^ 
^ given -in- [2]^ ' ObaJjHe th^t. the- JDMi; 
need for overlays or paging.: with res 
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storage. In the preceding; example; ^here was no 
need to dise an array. In addition the programmer 
is not required to have explicit knowledge of ' any 
physical pointers. In .the pivot example, the. pro- 
^grammej?' jiever sees a physical pointer; all data" 
'manipulation is dealt with on a logical level using 
-ihe logical structure of the schema. Each DML com- 



TAXZ SVAR OVr QT BASIS, .BSTAi 

'■' Qui JIKSK (E1CU» 

■ OUJ- SFM. 

cm JXSJC (acTr SHVAR. USW) ^ 

CALL Sm ii^ptfJd^ ZlCH, BVAS» JXFR) . 






•% ' . • " ; 

. ■.. auiCKD C3HC0I., care, inat) . . < . - ■ * 

15 (XETC. d- KEfW) CO TO 20 , 
CALL ISM ^(3IfflOtf,Jl»3l) ' I 

, 27 CJEA) 99940,999 ' ' - . ■ . . , , 

, 2C- cci CFx C3HVAL,. 3HRow» m-, inw) ^ ... 

. . CALL'STK C3HVAL, 3HR0W» 1 -/PIV » "lEW ) . . .* * - 

CAU. CDC mP» IERR) " - - - . *' 

2T1RAJX THRU, OPDATE ^ITO COLOMI, DIVDE ZACH CaLi->iUm T 9 T^t^l 'Br 'piTOT 

■ ouiccR {3HcaL, ixni, ierr) 

^0 CALL aw (38C0L*-KETC. IERR) . ; ; ' " 

' CALL c™ (3ErAL-/3HtXiL, i, ID©)^ - 

: ^ - -TAiV' ^ ► " .. ^. ' ' 

IT CABStrj.CTjITStjOnJ 00 to 35 
CALL DiW JfiRR) , 

- 6.0* ^ . • ' i» 

CO TO 30 

i , • ' 

35 • CALL SrX (3MVAL, SHCOL, HJOl) ' 



mjtxE^ 



SETOkiTfL rrt or this ccuAct zLoer's .boh tCiSi) ' 

3S CALL 3>N (3HB0W,' 3HCCL, XSIH) 
CALL CJCO (iHROtf, XTm, IDdt)* 

Tlenm -7. PlTOt vltb IXL 
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}90it\ xxftch no 118 pxvor mv 

fco OOL M»| <3w«.. 3iai«r». xwi);- 

son am mw Ait»(iiQ> z« tios coumt 

.. / (ofio. xriK)'co TO 70 
if <nw) €o;30,.g99 



90 OGRUSfOmOB UTHI * OSftX^OSB 

aiiL m isifkip knaoo, x«r. mat) 

on 9CK xm. Jim) 
CMiL A» (329tar, ^BC0K7. zsa) 

QLLL AMI ^IICOET. IXnt) 

OriUkXK TAUXAU KUMDCr . - 

70 CALL G7M (3nVAL. SBCQL. V. ISV) - 
¥ • V ♦ . ^ 

ir Cabs (w). tr. xraz^) co to Oo 
CALL on C3WAL» 3icsa.. V. isn) 

00 lO.CO \ , 

. ncxTg zxso ox'tawmcx 

So ooL zm C^ncoi., 2sa) 

00 OK TO jocr Boonr n mor kou 
90 CAU. SMC i3t»od, omt. imt) ' 
. ^' 9^ cm. TW' (3»tou-. IZ3»> 
IT <in«) ioo» to, Si^ 

nXD BDOCr VAU3X m .nfTOT^COLUWI 
•^XOO- CALL SMi (3lt30L, »TC,'inCF) ^ 

105 CALL m .(39C01.. zca) 

106 IT (loa) 2iO, 30. 999 ^ , 

Tzsour. 03W5X Piyor aotf iTssLT 
^110 c&LL 8CK (3ISKH, jera^ IZSS) , ^ 
120 ooz. c»« (3ttou. >t»c, M»> \ 
IT (mc «. nxp) CO TO i3o v 
call' C7M C3By»L. 3»«v. z» ZI3») \ 
• CAU. m C3HVAL, 3i»oi*» x/m, infe) 
130 CALL not C3BRoy. zsa) ^- / \ 

57 Cfl»).lto, 120, 999 ^ 




999 CAU» m'(zsa, 3i o) 

CALLAVW7 
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• - c «CI»»7- (coatimiatlo£») ; 

matKi' is a 9tatenieiit*lp.. terms .of the logical struc- 
ture and dt axctoaaaticaily execofces the chamber somfe 
physical details Ijnpli'ed in that statement. - 

PRACTICAL .CaNSXDERATICN' ' 

■ It is not sTj^sted that the foregoiiig suhrou- 
tibe he^ used in a practical- senSe. isince the tone 
or this ipaper is basically tutorial, ire have <^os,en 
the" flamiliar pivot ppera^ion. .fius a vehicle, for con- 
•veying.the fundamental ideas and methods of data- 
structtcring data laanipulation. The exconple 
''serves to illustrate all of the EML ccBunands whi<^ 



. are typically used^i. 'process ^of data hase icanage- 
ment * A ^^ajor" dtitficiilty with" the simplex nethod, 

. J.S that\aiCo3riginal2y .'sparse. iBatrix is usu£Ll3y auL* . 

' tered such that this *sparseness . vanishes We there- 
hy lose'»the*"tonefits:a data ba'ae mmageiaent system, 
can provide £x terms of 'storing and marjipulatiijB ; 

. sparse xnatripies. ■ - ■■ 

^Ptorn > practical ^standpoint the reyised sim^- 
laex method nar'nia^s certain weilrkncwn congnita-^ ^ . 

. tional" advantages ^^72* The .initial^ prohlcm -de scrip-. 

-tion is; not "subject: to modifloitionj rather 'than ; 
.explici-fly pivoting, -ve- succ.tf ss'ively update^ the ^in-, . 
verse. -of each nevhasis.. . T1m*» current ba;;fs 'at any 

\||tep may i>e. coxnp\Md *by;4^tking' the prcd>:^ct^ of ele- 
^msAary matricies-,^&re pciich elementary* matrix i^ 

: thSidentity except' for tine column. Thxis a data 
^Structure capable .of stzpporliijqg the revised ^simplex " 
method must be able to store both the original- prob- 
lem matrix and the noj^-trlviai;. columns of •elemen- 
tary matricies which compose the .jaroduct form of 
the basis inverse. 

' ' The structiire of Figure k suffices for the 
original iproblem. Recall that no storage is allo- 
cated for zero valued coefficients, ^o substantial 
storage savings arc realized as problems tend ^. ■ 
toward sparseness* There is also a savings in pro- 
cessing; if a'coefficient is not. stored then it £6X1- 
not be considered for^processing. These savings . 
result regardless of whether there exists a sp^M^ 
pattern of sparseness (e.g. band matricies) or ]|^-' 
ther there is no discernable pattern. ^That is','^H)te ' 
savings depend only upon.. the percental, of zero el^. 
ments and not upon their, arrangements 

• There are several ways to store columns of the 
eiementary matricies.' In situations where It is 
undesirable or not possible. to- store these in core, 
a da^a' structure^ similar' to that : of Figtare 8 could 
-be used. As tHe* structinr^ indicates, an opcuirence 
of PROBLEM has" mai^-<x:Qurrences. of EXPRESSION', of ^ 
VARIABIS, and of JOTERSE associated .with it. The 
former two descri'tiB- .the initial matrix, which is 
not subject to change* At the beginning there are 
no occurrences of INVERSE; however one occurrence . 
is created and added to_the set H^QDUCT" at each 
iteration of the revised simplex .algorithm.. "^In 
the. DDL the da.ta i*em type COI#UMfJ-VALUE is defined 
to be a vector. So a IML reference to an occur- 
rence of this - item will involve an- entire vectctt^ of 
column values 1 Far instance the command (IFW 

. ihmOLVy l^HEROD, DATA, lERR) will fill the array' 
DATA" with the data vector of values from COLV for 

*, the current member of . the- set 5R0D. The position 
.of a given column in itsr elemeittary ma^lx is 
"'stored SBjthe. value, of COLUMN-NO* Finally we must - 

. be able to acBfes the columns in the order in which 
iiilr respe'cti-ine elementary utricles were 'generated* 

' Since we ciji ^declare (i;^he'DDL) an ordering for. 

■ the'.meaibers ocT a set, -we speciiy- PRODUCT to..be or-^ 
dered^OTf^FIFO basis ' (First -In-First -Ojit ) . . .Thus, 
ac<i%^^^currences of INVERSE f or \,a.'parti-cular 
occurrraice 'of * E^OBLEM is based -u^xjn the w^der-in ' 
wfiich." those TOCurrences were originally ^added to 

_the EBODUCT set. «It . should be clear that the ML^ 

^ccBmands lused forJLaplementation of the revised 
sln^lex method in a data ba^ context are the same , 
as those presented- in ^he pivot example, (except;' 
for DSM, which 4s not needed for . the revised sim- 
plex method). 

We realize of course tfiSfe^he revised simplex 
p^nd the product form are"^ell.-laicmn^'and- tl^ nam- , 
eroos specialized data packing s chem es have^been 
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^ developed: Tor treatiiig spefccwi inatrjl^;tes What is 
' InteSre sting is the indepen^nt^ development of 'data 
. handling mechanisms vhich^ are, ^in a sense, general. 

packing schemes. The^, term 'gejieral'* refers to the ^ 
. ability of & single data base management system 
• ( e . g . ^QFLAN ) to support . data ( numeri c cuad non-^num - 
eric) storage and analysis for a broad va3:iety ap- 
plication areas." Past and present projects involv- 
•.'Ing the GPLAN data management system .include the 
'areas df -water quality, control* and planning,;. fore- 
stry, heaUh care, material. requirements planning, 
PERT maiiog&ent, auaid socio-economic research- Each*- 
^ea"reqiilre*s data structures thai; embody that 
area's re^vant characteri/stics. " Typically, eagl: 
area also requires that severeil types of. daalysis 
be pcrfonnable;-. In. the -water ^qu^-ity area for in- . 
stance , analyses of the f olibwing^ypes-.have' teen. , * 
f oymd - to be useful : ' • Simula ti ons , '^near program- 
ming, non-linear programming,, and various statisti- 
cal a na lyses fi3]* Progprami' torperform any of these, 
may. be implemented, with tte DMI>-. ^ ' \ 



QUERY LANGUAGE " . • . ; .': 

. ^ Where asL GPLAN/IimL requires thjit a user^^write^* ^ 
programs '.in- a host ^language with, the utilization 
. of pertinent DML commands, the GPLAN Query . System 
does not 'require one to" be a. programmer in order to^ 
utilize the data base for purposes of display, or. ' 
execution 'of large application routines. The user. 
\ needs merely to spwixy wha.t is to. be- done; there 
is ho 'statement of .the procedures to' fee. followed in'^ 
order to acccmpllsh"the 'ta,sl:.. ..Examples .of ^-.ycry ^sim- 
ple concnands ar e , 
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(COl«) ^ 
COLUMN- VALUE 
CCOLV) ^ 



riinrt 8. ^^imlbl*. Dat* structure for th« nrriacd Slsgsltx 
*? Stetijod vitb Product yora or \tn Hs^r^m. 



We lljave indicated in- the fcxregoing discussion 
6ow data base majiagement techniques aiay* be used T^y 
prqgrammers in LP develppmetrt, in such .a way that 
efficiencies of special'' data packing schemes are . 
realized. Similarly this saiae .data manipulation 
language* may be used to accomplish any other pro- " . . 
grammable analysis . Thus ^data hase .maiitfg^ent ' V 
tools allow a siiigle data 'base ,' containing both 
numeric and" non-numeric -data, to support requests - > 
of various users* for a variety of analyses" and re- - 
ports* An LP package "is insufficient for . this sort' 
of task, althorigh it may be quite efficient in what. 
it does do . 'On the^ other hand, an installation 
which' possesses "a data management system, can/ devise 
its .own- LP routines without regard to special pack-, 
ing scbeme'^fe and* boiff^ring procedures for interface ■ 
with auxiliary memory*" These arj^i^handled airtoma-"-- 
-tically by the data management ^system,,* and its gen- 
eral packing -sch eme provides substantial savings in 
the event of sparseness. , Anotl^er inportant feature .\ 
of the data .base management system Is its capacity 
to -support a query system that allows non-program- . 
mers to utilize j£ data base and cLnalytic routines. 



LIST 'COEFFICIEOT. SOURCE 'POR VARIABLE. ID = 'XI' and-. 
jtXJKSTHAIOT.ID = * \ 

■LxST VARIABLE. IB" AOT'CONSTRAIjrr. ID' PDR COEFnciENT.- 
SOURCE^= 'TEST,!'- 

Vppn^ receipt- of sujjh commiands, the query .system ' ' / 
"analyses the^arequest,: sets, up the- necessary "DNIL*^ com- 
mands, executes those commands and supplies the ■ re- 
quested" dafe values;. -The system is designed such 
that it permits' the selecti"'/e '^ (or, unconditional} 
retrieval of a,ny dat^ configuralfton. Moreover, it . 
permits execution of' application rou-tineo using aiv 
desired (and germane) data from the da'^ base. The 
fundamental-, query syrt^cuTis / • • ' ' 

<CCMAM^>.<51ETRIS;/AL CLAUSE> <COmiTIOIIAL CLAUSE> " 



The^ cbnaiiknd indicates which ^application routine is ^ 
to be -executed; in the\queries above, LIST indicates 
that a report generator is to be executed. In the. 
retrieval clause the user specifies what data are " 
to he used for execution; this retrie'/al is depen^ 
dent on conditions specified in the conditional 
clause . . ■ ' " ^ 

■■■ ' A user of the query language is allowed to pre- 
sent arbitrarily ^coarplex . r etr ieval gclause s • _ Not . ^ , ; 
only may this clause contain the - names "of « data items 
■ to be retrieved, butrarithnctic operations (using 

.literals or. data items) and both single and -multi r 
' variate functions may also be introduced • The con- 
ditional clause" is ccmposed of a Boolean expression 
which may contain data item names, literals, arith- 
metic operators, relational operators, logical cp-"* 
^ erators^, single-variable functions and multl^/ariate 
functions. The^query language' also permits the 
'use of noise words, synonyms and various^othe'r cos- 
metic featxares for the convenience of the -user. 
A conceptual overview of the standard GPLAN 
. sy.stem is portrayed in Figure 9. The library of 
'appll-iation routines is compised of two sections: 
standard routines and special rojitines *. "The s'tand- 
ard iihrary of applications consists of routines 
tt) generate reports an^* plots andl to:' perform linear ' 
Tegressions, "^rattdstical .analysesjt and data modiifi- 
' cation.' The" library of special, applications may 
'include such r outine s , as spe cial report generat ors , 
"linear SLnd. non-linear, optimization programs • .The * - 
issue^of Intcrraclng such ppt'i a1 zatlcn- rcmtix»s 
with a network -data base is discussed' in [if ^. .Oae^ , 
method of interface^ has been demonstrated in-.the * • 
pivot example... . 

■. PATA 'HAmiiNG J'QR MATHEMATICAL raOGRAMMINa 

In this section ve examine' the implications of 
data base management 'techniques -for those who im- 
plement' mathematical ^programming algorithms and for ^ 
those who .'make use; of such ^ implementations. In 
so doiiig,.- we ut^ilte the distinctive GPLAK. features 
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or the OfitvcxrisL. dkta base slxucture .the lAnguage * ^ 
for non-progrtnnjesr interface ifitb a data base (DML)- 

"andvtbe 'qyjjery laigiiage^. that allows ' non-prograunners 
to ©IXectlvely use^ a data, "base and pertinent appll-: 

. cation routiaes. Not only does the GPLAN Frame-^ . 
work allow ror the obvious, i.e., the Solntlon oT 
linear and non-linear optimization problems;, it also 
adetresaeo the Tollowing considerations, vhich nay 
pe^ps .be -moiie s^ibtle, btct axe certainly ^.Qf prac- 
t'icai^gnificance [51w \ v* 

f iV Tte resolution of erronechis formulations. - 
2! Trcatznent of coefficients vhich arc them-jj 
selves functions. ■ ■ , " 

: : 3.. Situatix>ns wherein matrices ' contain-com- 
iDon diawL* '. 
' ■ U. - StoraSiV^ sparse matrices^. 
. 5^1 Utillzdffion of data to prodnce tiiaely, 

non-^routine reports', other than the repdrt' 
:fumished by a general mathenatical pro—. 
•^^ grannaing routine. " - * 

6l. Ability of a' data base^to support other 
' varleiies of application' routines (e.g., • ^ 
-siJsalatiCTis, regressions, etc.) in a;ddi-^ 
^ion to nathematical' programming. 
- : . ' '. ' ■ . • ' S ' ' ' ■ 

" ' ' The modus- operandi f or ''effl ctlve a<ycanoda- 
' *tion cif each of these attributes may be/found, in 
rh']. -Only a . very brief treatment can be .given here. 
With respect, .to the .first point, when erroneous co- - 
effid^s or Inqproper farmulation i^ suspected the 
xeaiay availability of informatipn vlth regard to co- 
-Affident sources iod currency is inportant. This . 
^"^as obvious' implications for the' way in ifhich data 
. is orffLnized.aM rfitrlevie^L. In this -connection the 
query system providbes .a convenient tool for data 

•editing. • \- ^ 

Concerning the second attribute, it is not un- 
. coamon that coefficients are the results pf func- 
tions that have been evaluated on the basis 'of some 
. gtlyr data. There may even be alternate functional ' 



forms ^orr alternate data sets .JTor evaluating a func- 
tion) which suggests the- need for a facile methodv • 
of i'nterfalcing- desired functions 2*lth the math^pro- . 

' granning. routines which :depend- upon them. Indeed. 
•a linear programming routine, may viewed as a 
func:t.ion that requires, (jrecurringj evaluation in ' " 
order to execute a non-linear progranniing routine- 
r6l One technique for handling this 'isituation in- 

. . volves an "elimination of the distinction between . 

.•^ data 4nd function,, with respect 'to ^ta base .con- ^ 

' struction;fi.i.e., functions are treated as , data and , 
included in the !data base structure . . ' - / ■* 
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Attribute numbe:^ three is Important for ca^es ^; 
rhere there is interest in several matrices, which 
_ire not entirely distinct with respecf to con- 
straints and variables. I For ««tnipaf>we may be .in-, 
vestigating a problem which tes muKiplc plausible 
farmulationsV-Sone pairs of -fttich share . constraints 
(and therefore variables^. Care must be taken to 
assure" that updates to constraints in one mateix 
are reflected in other matrices vhich share these 
constraints^ this i^- not a triviaiujatter where _ 
large volumes of data arc involved. Data struc- 
ture that^i^m support this .situation is. given^ in 
Figure lO^^^^^s structure allows us to store a. ' 
constraint' bniy once, an* at the same time indicate ■ 
that it is to be included In- an arbitrary .number - 
ar-matrices^ This avoidance of redundancy averts ^ 

potential for inconsistencies. A de tail ed 
description of "thris structure at tbe occurrence 
level may be found [Uy. Figure 11 di splays ^verba- 
tim eacajaples of the kinds- of queries which could 
be sulinitted, if the data structure is as shown in ■ 
Figure 10. The first .three- queries^ show how que-_. 
ries may be" phrased to locate and rectity errors in' 
the^ data. The. next two queries demonstrate ways in 



. vhlch^ are LP" pr pibleiiui- may be formulAl^eid for execu- 
tion. The ilnal query asks for^ljafiic statistics on 
the cpefticLtv^ v»l\ies of vaplabie U97o in natrlx 7-' 



z^catmjiin.'mt^cwr^iBrf.yFS^ avd coEmcinir.sounct tor vaxubu<xd.< 
' cwaol o MJijSiuf .mag s> ifio. (connnc2iiiT'VAws)*x.ii». b' tAmjbx*iP * 

r " . . ■ ■ * J . ■ * 

a^kir COBTKZK.IOIIS FOB TUaJUS2*2S »-'t»9T,4M>.M%2RpC.XD - 7 



' »any 3real-warld applications entail the utili- 
zation of sparse xoatriceiS. In the effort to avoid 
. storing .2«ros, many s^iemes have been devised for 
packing (and unpacking) non^zero coefficients .'^into 
arrays. As already shown in^tbe pivot example, the 
GFLAN . concept allows a^ matrices (sparse or other-. 
mise )i to be -acccxaodated by a single simple logical 
structure, whjth realizes sufistantial storage sav- • 
ings if a-.matrix happexis t^o be^spar8e• Only noh^ 
. zero coefficients need to be stored; and storage 
'space is neither used nor even aillocated for zero 
•coefficients*'/ . " ' 

I^svBnab2y managerial decisions are iiot' based, 
^soley upon the output of mathematical programpiing 
routines V The fifth consideration, indi'cates ^he 
need for a facility ^o generate other reports 5 
£1*001 a, data base and l^quently these are nonJstan- 
dard in' terms of the types and configurationai of 
data that are retrieved* ' The GPIAN ^uery system ' 
ap.ows the selective retrieval of - any configuration 
of data as a result ^f typing ah English-like, . / 
■ Jion-prooedural ^uery at a cociptfter -terminal. This 
obviates the necessity of •writing a^jpcrograa every 
tljDe 'a new type of report is needed* ■ This, con- "'^r: 
cept can be extended to include not. only retrievalV 
but also the ekecutiop of laz^ge application rou . 
."^nes that' are '^not of the .^nathematical- programming* - 
variety, Farthermore , su^ cxecrutions^ can be'accscjn- 
pliahed through. the. qiiezy language. The.Tesult. is 
a situation wherein a netva£lC''*^ta base can suppccrt 
a broad spectrum of - analyses for both* programmers ' 
and non-prcgranming users* 



needs, in circimkstahces where flexibility is unim- 
poirbant and efficiency is paramount. jg- ^ 

ItLstoricaUty, the -tarend-haa -been for each math 
pyogr a mnl ng^ Igplementor t^> .in.effect', devise data' 
management routines. ^We submit' that advances (both 
. paat and future)' in the data base management field 
^ can jrovlde valuable concepts^ techni<iues and* tools 
' foe ;the (Istta handling aspects of mathematical pro- 
-■■ gramming^. ' ■ 
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The mi^cr inxent has been to introduce funda- 
rvental concepts Atcdi the realm of data base manage- 
nent and to sisggest their potential for contribution 
•-to the solution of mathfsmatical programming problems.' 
This' contribution-ia considered with respect to both . 
.the liapienientars and users, of mathematical program-^ ^ 
ming algorithms. , GPLAN, a generalized data baJie man-^ 
agement and query system, was descrj^d* It is ^ ^ 
essential to observe that^there . is* invariably a 
tradeoff between specialized and general' systems. 
The former are genially mere . efficient due to t^lr 
liailfced flexibility; the. latter are typically nq 
as' e^icient 'due to tbe£r ^at flexibility. 
jj3eirficiency and inflexibility have costs* 
provides a single mechanism vhich be. used 
treat the spectrum of ap^clal cases and to ' 
a variety of appUcaticSji and users vcorking 
the sazae data. base* As an exercise^ this 
ayrtem could be tailored to meet only speci 
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Abstract^- Recently, modifications of the methdd 
of ''multipliers haVe .been proposed which do not 
require «xact; minimi zatiop in the intermediate 
xinconstrained problem • . It can be shown that any 
Lagrange multiplier . updating scheme which yields 
quadratic, convergence with exact minimization 
retains'- this rate if only two steps of a Newton 
iteration are-"- taken. It can also be shown that a 

■quasi-Newton. iteration in conjunction with the 
> multiplier update proposed by Tapia yields a 
superlinearly convergent Blgorithm- If the 

' penalty constant is allowej: to increase to 

V. infinity .at a cei^tain' rate'i! simpler multiplier 
update formulas will exhibit good copvergence ^ . 

>rates. - *■ 

-Recently there has been much research, iii 
' solution of^ constrzLlned- optimization problems hy 
the method of " multipliers- This method was. 
first proposed by Hestenes [1] and ^by Powell [2] 
independently in. 1969. It involves solving the 
■ problem . . " ■- V ^ 



mxmmi ze 
such that, 

and 



g(x) 



(1? 



where- fz R" R and g:. r^ r"- by ' 
cessiVeiy ininimi zing 'the augmented Lagrangian - 

. . Ux/X,c>-f (x9+>L'^g<x)-»^cg(^x)'^g(x> ' - 

in X.* If. X* is the-. solution of prc^lem (1)' 
'then there exists a unique A.*6 R* such that: ^ 
7 L(x,*,X*,c} where . V " we meair the " 

^ ^adient taken with rei^pect^to the vari<S?le x,* 

We will assunfe throughoilt that . ni '< n, that - 
.^.7g(x*) is, of full xanJc. and that" 7^1.(x*#X*-,c>. 
^ is .invert ible. It can be shown. Bays. [31 that 
^. ' there exists a consjiant • c° such' that for' 
i c > c^,.^x*' is^itn ahC9nS'trained locaJ, minimum . 
-of LCx*/X*,c) and ^or 'x-'i^n a. *ieighborhood 
• of A*, ^L(x,X,c) has a local minimum-in ' xT^' 
•• • • • ^. . . ■,' ' » ■ 

\ The methda of multipliers procedes as - 
, follows:. ' ■ , ' ^ . \ 

■' \ Ste^, V Choose- an initial approximation, • 
■ ' • X° to ' i*, and aii^iri^tial c . ■ 

.•step 2: Choose ^ tp be the minimi zer / 
" ; of L(x,X , c*^) ^ 



, Step 3:- . l>et 



^k+1 
,k+l 



P(x^,X^,c^ 

„; k ,k . k+1, 
•U(x ,\ ,C' . ) 



Step 4: Return t6 step 3, with k * k+1. 

The function U may be referred to as a nxialti- 
plier update formula,- a*he .function P is some- 
■ times chosen siinply so that c- is large enough 
that a minimizer -of^ * i* exists,- while for other, 
schemtes tends tq infinity near .the solution. 

A number of multiplier update -formulas haye 
been proposed . we consider the - following 5 



U^UftX,cy =» X+cg<x) 



(2) 



T" 2 ' ^1 
U (xA,cr =^ X+[Vg(x) L(x,X,c)7gJx)] g.(x) 

■ - ■ " ' \. . . 

UpCx,X,c). « -[7gCx)'^7g(x)]^Hgw'^Vf'{x) (4) 

U„Cx,X,c) « I7g(x)^7^(x,X,c)"^7g(x) 3'^ 

T ' . " X 



Cg(x) - Vg(x)'^7 ^L(x.X,c)*^7f {x)3 -cg(k) 



Formula (2} is the one originally propose by 
Hfestenes and by Powell-.. Rupp [4], and Bertsekas 
[51 have shown that for c^ . suf ficientiy. large 
the Hestene.s-Powell update gives arbitrarily 

■ good linear 'convergence . Formula C 3) was proposed 
by Buys [3] who showed that, if Is considered 

as a function of X, (3) is equivalent to one 
Newton iteration on the problem 'gCx(X)) = Ou^ 
and -this gives :.guadratic yconver^^^e . Formyla 
^4) is jus't the projection of oi^to /the " 
linear span of . 7g^ (x) , . . . VJi^Cx) • Its use has 
been prasposed by Haafhoff ilnd Bujrs' Miele 
et at [7], and Fletcher [8]., ^Formula (5) was 
originally .proposed by Tapia 193 "is a consequence • 

.of a "geA^ifSCi' theory for solving constrai^d 
problems. ' ^ .... 

'k k k * 

Finding ^x ' w*iich' minimizes- L(x,X ^c J/'. 

exactJ^ is of coiit^e" impossible^ and even finding . 

a good approximation may be. very 'costly ♦ It is 

•natural to consider a3,gorithms involving s^n^jle 

approximatations 'to the minimizer. ■ ^ " 

^ertsekas- [53 « considers an implementat^Lon 
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with the. Hestenes-Powell update where the minimi- 
'-- zation-is--carried-o«fc until — 1|-7- L |j- is- less than — 
some tolerance, and shows that for the tolerance 
correctly chosen one may obtain dbn verge nee as 
good as in the case of exact minimization although 
nothing may be said" about how difficult that" 
tolerance is to achieve for large penalty 
constants. • . ' . ' . - - 

•■■ : ■■ - ■ : 

Another strategy is to take a ^mall numb^?^ 
iterations of a minimization algorithmr ^ strategy 
referred to by Tapia as a diagonalized multiplier 
method 19] . Such a strategy has also been pro- 
pose>i by Haaxhoff and Buys [7] and' in^lemented 
by Miele et.at {6]* 

We consider first the case of algorithms ' 
implemented with bounded penalty constants i.e- 

^ c for all k'. If a multiplier update 
formula such as that of Buys is used one needs 
second order information about the functions 
involved so"* it is reasonable to consider methods " 
in, the unconstrained minimization phase which 
generate and use the second derivative or an ■ 
approximation to it. 

k+1 will refer to an algorithm where the point 
X is determined by j steps of some iterative*^ 
algorithm as a j step diagonalization.^ . 

Based on different considerations Tapia 
C9,ia,ll3 proposed an algorithm which is equiva- 
lent to a , one-step diagonailizati on using Newton's 
"method- That is; the point is determined by - 



k-1 



r — 2_ , k-1 -k-l K.-i- -V ^ ^ 



■1 k. 



and 

vk 



.\ is chosen according to Tapia *s update: 
X'^ « U(x^" ,X,c^) where ''x = U Cx^"^) . Tapia 
shows this method has the following convergence 
properties (11) - " 

Proposition 1 . If the one-step diagonalized 
jnethod of multipliers is implemented with Tapia 's 
update and Newton's method, the sequence {x ) 
is uniformly quadratically convergent to tlje 
solution X*". ^ 



' update, behaves under diagonalization . If we 
----consider the - question of quadratic convergence ~in" 
the space of both variables, x and X we* get the 

. -following result. 

>■ 

.• .Tprdposition 2 . Considei^ the method of multipliers,. 
■ *'^-i^^>^ement^d with a " given multiplier update formula" 
•and vith, bounded penalty constants. Uniforsi^^ ' 
'^quadratic convergenl:e. in, (x,X} occurs with- 
exact minimization; if .and -only if it occiirs with 
a two-step diagonalization using Newton' s- method. 

Proof . See [123*' - ^ . 

Since both the Tapia update cind the Buys 
update gi'Ve quadratic convergence for exact 
minimization, it follows then that one may get 
the same convergence for a two-step diagonali* 
zation with Newton's method, and for such update 
it seems nothing is contributed to convergence ■, 
rate by further iterations. . " , 

Since using Newton ' s method required ^ 
knowledge of the second derivatives of the 
f"»nctioi^s involved and requires a large number 
of function ev2Quations at each step it would be* 
» advantaigeous if a quasi-Newton method could be 
used in place of Newton's method in the deter- ^ 
mination of x^. If this were done with one of 
the quadratically convergent- updates slich as that 
of ^ Tapia or Buys^ it would be necessary to use an 
approximation to 7^LCx,X,c) in the multiplier 
update form^la. Consider then the following^, 
iteration: " ■ • * 



, k+1 



k+1 



k+1 



^k+1 



[ Vg U^) B^^V,5uui^}] [; g Cx^) 
-Vg(x^jS^7f (x^)]] -cj^gCx^) 

„/• Jc+1 k „ k+1 .k+l Jk, 
V(x , - x r^jg^C^c ,X ,c ) 

_ _ , k .k+1 k, „ , 
- 7 LCx ,\ ,c ,B ) 
X ^ k 

^, k+1 - k+1 k, 
P(x - ,X ,c ) - 



(6). 



This " res lii-t is "obtairTed *by~ noting~that 
Tapia 's algorithm is essentially equivalent t,6 
using Newton's method on the problem 
7^L(x,A,c) = 0, g(x) =0.^ 

. By uniform guadratic convergence of an 
algorithm we mean there exist constats 
M, V > 0 such thait if 11 x^ ^" 
II xk+1 - x*J| M 11 x^ - 



^ X* II < v., then 
^ for all k. 



The notion of uniform quadratic convergence 
is sJ-ightly stronger than Q-quadratic convergence, 
but almost all iterations which are Q-quadrati- 
icaQly convergent eire uniformly quadratically 
■convergent Using the projection formula for % 
allows us to get uniform -quaSra tic conver- 
gence-' If we let X = X^~^ 
•quadratic convergence of 



A^~^ we only get R- 



■ Since both the Buys multiplier update 
formula and the Tapia formula give quadratic 
convergence when x^ is» obtained by exact. « 
minimization iti/is natur2d.f to ask how the Buys 
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^ The function VC'sVy,B) is the I^essian* update ^ 
fimction for some quasi Newton method. The 
analysis of" such an algorithm is somewhat 

, different for each possible Hessian update 
fuhction but Tapia has proved the following^ 

• result for standard quasi-Newton methods [11] . 

Proposition 3 . Cortsi^der the method of multi- 
pliers impleinenteca as in (6) with bounded penalty 
constats and with^ the^-func^on V beinj ^y one 
of the Broyden, Davidon-Fletcher-Powell , Broyden- 
Fletcher-Goldfarb-Shahno or Powell Symmetric • * 
Broyden, Heseian "update functions • The sequence 
of pornts (x?^,A^) will then be locally and 
superlinearlyi»convergent to (x*,X*>. \ 

" ^ ' , " •> 

V ^ Tteis result shows that the algorithm 

oih?*2?5iQd in (6) a relatively efficient 
strategy. It i3«^s$ible that there would be . • . 
some advantage in taking . more than one qiisisi- 
Newton step in determining x , howeve^ by this 
result* the convergence rate would still £>e 
superlinear. : • ■ • 



193 



It seems ^4ikely that if a quasi-Newton 

.m^thod_^re„useA with..t^c.3uys...upda.te .-iii-SQine L 

diagonalizod scheme that super linear convergence 
would result. / However*, it has not been possible 
to prove this^ y.et. It is . interesting to note 
that according ^to the above result, the matrix 
Bjr works well^'as an approximation to the Hessian 
in the multi^piier . updaj:e formula even though it ^ 
-is knovm- that' ^ ... 



Bj- does not converge to the 



Hessian in general. 



The above development seems to- indicate that 
-^..dae- or- .two-.s-tep-dragonaJLLzation^ u^-.as .effective-, 
as- exact minimization in ensuring fast con- 
vergence . Alternatively if one is using a 
scheme demanding a certain degree of exactness-- 
in the' minimizer^ tlfese. results give some . 
indication of how much effort is required to* 
achieve this exactness. 
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We. will reJer to such formulas as local multiplier, 
approximation formulas-^ Both the projection - ' 
update .and the Tapia update fall into this class . 
Note that it is essential that such a formula 
have the property U(x*,X*,c) = A*. 
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DIRECT APPROACHES FOR THE MINIMAX PROBLEM 

A.R. Conn . 
Department'of C^pmblnatorics and Optimization 
University of Watjerloo 



Introduction . ' ■ . " 
• • 

^ Consider a system of m real, twice contin- 
uoiikly c&f ferentlable* and in general, nonlinear, 
functions ^ * 

f^(x), i c [M], 

wflere . [M] - {1,2,3 — ^tr} ' is an index'set. Let 

. *M.(x) * Max f ,Tx) , . ' (1) 

* ■' * r i"^ 

/ X ™ 1 ' ^2 * " " " ' ' 

The proble^ under consideration is to lo'ca'te a" 
point- Xq such that- »j " 

^r all points x, at least in >a neighbourhood o£ 

^r*^' The ob'jective function M-(x) has discont- 

fnuous Cirst partial' derivatives at points where 
two or. more of the functions f , (x) are 'equal to 

^he maximum. This is true even if the f . (x) 

< ' ^ 

/have cotit:^nuous first partial derivatives. 

^ . As an •illustration,.', figure 1 shows the 

contours for M^(x) for a^ purely linear case, viz. 



■ ■Example I > 



f ^ - - lOx. 
f ^ - - lOx 



si 

2i.5x2 



• " . f , - - 10x4 :+ 20x^ - 40. 

*The points of, discontinuity are indicated -in 
figure 1 by dotted lines. 

W4 first note that because of the discontin- 
uities in the derivative the well- known gradient 
type methods canxibt be used directly to minimize 
M^(x). ^ ' y , ^ 

Secondly, we note that c^a equivalent formul- 
ation, of the prpblem P is (see for example' [17]) 
• . ' . Minimize z i -p* 



*su9h that, z -^f^U) ^ 0, ±' c [H], 

where z is a new" independent variable. 

But in general, ^ is just a "standard" non- 
linear programming problem in nrWL variables, and , 
c^n Be solved accordingly. In fact, .this iS' 
exactly the approach taken by [16]^ 

'Thirdly we 'note, (a^ is. easy to show, .on 
•applying thfe vell-toown Kiihn-Tucker condition^ to 
P*) that a twf'T%4ma-r optimum at- Xq can be character- 
ized' (see,^ for example [17H by 




Figure 1 ^ Contours of M^(x) for £3cample 1. 



i€E(xQ,0) 



u^7f^,(4^);..o. 



(2a) 



(Vf^(xQ) 



3f(V 

^ ax, ' 



• ax 

SI 



•J ) 



and 



wh'ere 



icE(xQ,0) 



u^ * 1, 



2 0, ± c E(Xq, 0), 



{2b) 
(2c)" 
f2d) 



E(x,er- {i€[M]lM^(x) - f^(x) .< e}. 

, . • ■' : ' t 

(i.e., E(x,£:) is the index set of those functions, 
f that are within ^ g of M-(x) at *x). 

^ We shall now fonnula"te informally an approach 
to the', minimax problem based on thre^ propertied * 
cited above* , " 

Suppoae^e are at ' x-. Furthermore, suppose ' 

. ■ ' 

that f^(x ),'i ■J.,..*,q take on the maximum. 



function value M (x ). As in most opciiiii'zacion», 
approaches -ouj^ aim is'/co determine a direction, 9 , 

say, such chat ac* x*^» decreases in this 

direction'. It then remains to determine how far 

CO. step** in the chosen direc tion. and the iteration 

• • is*defined. ' Ic. is clea'r xhat since the f,, 

• - ' ■ ' k ^ " 

i l,....q are less than (x ), locally at 

least*, we are only conce/sjed with those functions 

k . ^ 

f (x.), i.-'i,...,q. V^e shall terra these functions 

tfie- active functions . ' 

Let us suppose we find a direction d such 

tl^at f , say^ decreases^ If it should happen that 

in addition, f^^^^.^f. decrease, then we have a 

descent direction for . However, suppose that 

although f^ decreases, increases. Then, as 

is obvious, will in fact increase. Yhus vh^i 

we must do is dec^rease all ;the active functious 
simultaneously. . . / 

•If we not^recai^ the non-linear 'prograr^r • rr.-. 
^ formulation of the pfoblemi, we observe that .uLivt> 
functions correspond to active constraints ar.c 
that inc/easi<ng» for /example , while 2 is 

dec reading ^. amounts to violating a constraint 
"(assuming was an active function at the start 

of the iteration' and z |" M.Cx )). Thus a' 

sufficijent condition that we have a suitable 
descent* dir«*c£ion. for^ M. is equivalent to 

. ifcnsurin^ that no active yconst r^a in ts '-are violated. 
^fThat this is. not a nee essat;y condition we will 

/ee shortly]. This sit/jation is^ a . familiar -~ 
. occurrence in^ the conte:tf't oX-;jitfth*ematical . ^ 

pj^ogramming (see; for" example [18] - and [15]) and . 
•one approach to ^resolving ahe difficulty is via 

projections. [See [8], for further references and 

details ] . , " , * - 

Xn ^elementary exposition on projection 

matxicesjnow follows:. Sup*c^Dse we have the 

fbllowii?g b'asis for E : ^ 



where 



V I sq q+1 , n 

10 



Consider the 'fallowing n^n matrix, 

~ ( , T -1 T ■ 

P I - N*(N N) N , 



(3) 



where N= [a^-.-a^], (i.e., N.. is an n^q. matrix 
of rank q) and 1^ is the n^n identity matrix. 

It IS easily seen that Pa = 0, Pb = b Vi 1. 

2 • .J i . i '-'^ . 

Furthermore, P =J P ^and we note that" P c^kes^ - ; 

any vector ^in i into Q"^, where Q'^ denotes. the 
space^ spanned bv . * 

Now s^uppose we • take P '^Jith 

'a^ - V4)^(z,x) .i.= 1,...,(^ ' 



where 



IS ^s 



0' (2,X) - 2 .f (X) 



and is ined in -equation (15) below. 



[i.e. » 0 are the inequality constraints gf 

- * ] -r— vfe~nore-thatr "a^V and~~thus "'"P , "Ts"' indepbriden t 
\of^ 2. Furthermore, .let us define^ d e m"*^^ by 



where 



d Pe, * 



e « ;[1,0,,..,0]^, ^ e [R^t^ 



(5) 
(6) 



. ■ Now, in the case vbere the f *are linear, we 
have, by Taylor's theocelm 

<j^l2+Xd'^,x+Xci"] = 0^(2, x) H-nd*^ - A[d"*]'^.7f ,(x),(7) 



where d 



[d'^rd ] , .d'^,A c R^, X > 0/ 



But, by construction 



- d"^ - [d']'^Vf , (x) « df7(>^ 



1, . . . ,q. 



Thus 



C>.[2+Ad ,'x+Ad~] ^ ❖^[2,x] i = l,...^,q. (8) 
More generally-, for i '= 1 , . . . ,q, we have that^ 



f o: coocave f 



<;> [z+Xd , x+Ad"] ^ o.(z,x) 



1 q. (9) 



That is, in the case where the active functions are 
linear or concave, if the search direction is ^iven • 
by <5), no active constraint? in the equivalent non- 
linear ptogramming formulation are violated. 

Furthermore, sincere d = - I I Pe | | < 0,, if Pe ^-0 
[which we may assume, for the present], ' d is a 
descent direction for z. . Thus oi^r obj-ective is 
«:eal i2ed, i.e. , -we are reducing M^. 

Referring to figure 1, J^pt us suppose we are 
at th-e point x° where f, (i:)*^- f-(x) -•M^(x). The 
corresponding' descent direction for M^, "d~ is 
indicated on Ithe figure. We see that it'is an 
ideal choice. * In particular, we kee that, locally 
^t least, the indices of ' the a'ctive function?; do 'I 
not change. 'This result, of ^cogjse,. is an 
inariediate conse^*i6nce of equa'tion- (8) . B 
|Jn practice, not all- c^e f ^ 's i = l,...,q 

nee'd to be put .into j:he projection -an^,' in ^ 
particular, ^ im possible to er^s^te that Pe ^ 
and thus firnl a. descent direction that does, not 
violate thos<^ constraints, not iti the projection, ?- 
unless optimality has Already been achieved^ That 
this'is so is a consequence- of the.^eqaations (2a), 
.(21b)., ^20 and' (.2d), [A point corresponding to 
suih a situation is indicated'by .y on figure 2 
belrfw. That Pe' = 0 in this tasc is 6asV to' see 
since f^(y) - f2(y) = f^(y): We can, in'fact, 
jemove 7c>^ from - N in this case to obtain ^ 
descent] . - * ^ 

We. have not^d above that ensuring no active, 
constraints are viola.ted . in • the corresponding non- /' 
linear programming problem jls* a sufficient 
conditio^ for a suitable descent direction for M^. ■ 
^, As we will now indicate,- it "is not a necessary 
^condition. Furthermore, in such* a case ^our solycion' 
'to the problem of finding a descent direction to M ■ 
via the projection matrix P is still" suitable. ^ 
Suppose our active functions are f and f 

• ' . - . 1 ' 2*' 
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f2(x) 
f3(x) 



2 • 2 ■ 



2 exp C-x^ + Xj)- 




Flggre 2 



Contours of M (x) for Example i2. 



"2, 



both convex functions. That i5 q 

Choose as in (5) - The 'corresponding ' 
:;esuXc' to (9) is^ 

■ , (>^(z+Xd"^, x+Xd"] 5 (J>^(z,x) i - 1,2 (10) 

•tSlnce ^ <) 'is ndw concave] . 
However, by construction 



and . 

■'■ / 



ed* <0, i.e. d <0 



.and 
i.e: 



d y«f)^(2vx) - 0 

d ■ - [d ] 7f^<x) - 0 
d± rd']V2^x).--.0, 



I'Vf^Cx), - [d- j^7f^Cx) - d"" < 0/ (11) 
' < ^ ^\ ' " . 

or, in other words, locally at least,, both f/. and 

■ * • ■ ■. ' L _ * - ' " 1 1 

"f^ ,have decreased^ i.e., d, is a descent. 

direction for M^. ' , . . - 

Remark > This* in itself indicates . that the approach 
we take here is more general than just solving the 
problem P*. 

Furthennore, we remark at this stage that,^ 
since a majpr part o'f our algorithm depends upon 
determining P, we can make use of ^ the numeri<ial 
results available for determining orthogonal . 
pr^ections. ^ V. ' 



1,2, 



Having determined the search d i-rect ion »Wi»'' note 
..-.^^_^J? .^iP.^S A ii?e.:.^r_aJfunctioa..SA/cl).,as... 

M^(x) are likely"^ to occur at tgose points d.f which 

the changes in the first derivative cj:e dlsconfin- 
. uous^ [for example, in the linear case 'this is ' - 
guaranteed]. Thus, ^s an initial estimate* ^he 
linear; search approximates a subset of these 
points of discontinuity. [St;ecif ically that s;^bset* 
whichi corresponds to the points of discontinuity 
given by f ^ (x) - f.^(x) (where' j is one of? vhe 

active functions at x^) over ail r £ cotresiJnding 
■ CO inactive functions at x^] - If She apprcix^- 
. tion so determined gives a point .where the Actual- * • 
value of is sufficiently lower, then" it is 

'accepted - the line "search is completed - oth^r- " 
wise a more usual (and more .sophistit:a ted") lih^^ '. 



search, is required'! 



2. 



The algorithms of Bartels\ Charalambous ai^ ti. 
. Conn [3], [5]. " T^. 



The basic ideas behind chese two. algoritKits 
were those given 'in -sec tion 1 above. As^ originally 
presented, the first reference considered only.' 
linear functions [actually, piecewise linear -since 



M^(x) 



Max I a ^x' 
i 



and *the second ref^reivie 



concerned itself with only ion-linear funct^Soris f., 

- i.e., no attempt was made to fmpfove the '^-'^ "■ 
efficiency by making use of any linearities- f 
present amongst the f^. - 

We will attempt to combine the results of , both v 
algbrithms in one algorithm for the problem F.v' In ' 
addition, differences over the'originals willi'iJe 
indicated, as well as relating the results coi'the 
work of others in the f ield ,in a later section.'^" 

. *Thus we vill use the formulation P* .tc^-^' 
determine the search direction, but will then apply 
a related direction direc tlv to the function- ltc<x) . 

■ In the algorithm, each iteration conslst;^>£ ~ • 
.one, or sometimes two, directions.' The f irst *^ the 
one that»is always present, is termed the ' ;r 
horizontal direction.- It tries to maintain V'" ' 
constant^ the index set- of near active functibrrs. 
Two or more functions are considered near active 
if Chey are equal to the present maximum up {\ a 
specified tolerance. In addition, it .-is req^ed - 
.-that-the horizontal direction be such as to ^ 
decrease M . This -corresponds to the dire'ction d" 
in section 1. T>ie second, the vertical direction, 
amounts to attempting to satisfy the near active 
fijnctioiis es^ctly by means, of linearization,. -This ' 
is done ^y determining the least squares sblution 
of". In general, an undecd^t£ftmined system of'-r^ 
linear equations. As such this step is closely 
related, algebraically, to that -of determining P 
and oncfe again Is a well-worn patlT in the fi^d of . 
numerlcSX analysis. The necessity for the.-^rtical 
direction is given in [5] and details, for. the 
computation of both directions are se\ *out beiow. 
We. note that, in the limit these two directions are. 
orthogonal, hence the terminology. 

A linear search follows, that incorporait^^- • 
several simple features of the algorithm :an<?* 
tfumerical results to date indicate that the 



resulting algorithm is very efficient. 



2.1 Notation arid Orthogonal Decompositions '. 

. ,_^....j'.„,Whenlwe .w4nt„to .£ii3d.,.the direction- o£...searchJ._ . . 

ati-a' point we set z'"- 2- - M^(x*^). By doing ; 

so, a*near>ac&ive f unccion/in the .minimax^problem ' 
^ ,^ corresponds tfo. a near a'ttive constraint in the -non- 
linear prograciniing problem, viz, ". 



(12) 



We note that Etx »g) can be copsider^d dependent' 

k ' * k ■ . « 

only pn x ■ .^ince . M (x )7 Let ' 

where LE(x is Che index set for the hear- • 

active linear constraints.' - * ' 



Furthermore, in order to eliminate unnecessary 

?_".^sequen t i t era t ions., c ho ^e 

columns that correspond to linear functions^ f (x) 
will be added in ^first. - ' , ^ 

Ac this point let us accept the following 
convention; if, in a particular context, ^here 
will be^ no- ambiguity, -we jnay denote functional 
expressions dependent uppn x - and sometimes .g 
more simply, by abandoning one, or often both, of 
. its argument^. Thus , LIKx.c), NE(x,e) , qCx.c)', 
*'E(x;e), etc. , will sometimes be denoted by LE(x) , 
-NE<x)„ qCx.) , E(x), etc . , .or isomefimes hs ' simi3Ly as 
I.E," NE,,q or 'E, etc. 



the 
r 



2^0 -(z,x) 3^ (z.x) 



^-v. - lr, -^vf^(x).r 



\E(Sc ,e). (14)' 



ax 



(15) { 
' (16) 



2.2 The Algorithm 

S't^p O' i Set -Label ?«,0, k - 0; vs - 0, x - 
starting point; and'-a value of c', and .cpstop. 
•^^^ -^raax' "^raax "^^^ che;linear 

seatch algorithm'.. • It denotes an upper bound' on the 
^ ^dmis£ible stepsi^ze. (Note that Label is used tb' 
indicate whether a vertical direction should be 
taken (Label « 6) or n^t (otherwise). Similarly /• 
vs "^ indicates whether rhe vertical step was 
siipcessful (vs = i) or not (vs « 0)).. - 



Step 1: Set z, 



1^ 

"k ' At the point x*^ 

determine /the activ^ *f unctions Within the specified 

tolerance.^ In other-words, determine LE'(x^,e)., *" 

NE(x^/e) and hence E(x^^c j . if 
Step. 3. . . ^ 

■St:ep 2 : -DetefminE the projection matrix initially. 
(This step involves inner iterations). 

Step 2a : 3et j *= 0, * ^ 

• Q '.I, iThe n+1 x^h^T unit matrxxT 

- A = <J), (tTie empty set) 



We note that 7.<>j(z;x)..^^ indep^dent of z. 

e » y',0,:..,0']^ (- VzJ). . ^(17) 

If the n by. k matrix^ A has independent?* columns 
it cdn iJe' factored ^Itito tfie -form * * 



k«?^.Q, go^.to 



(18) 



^ where Q « [91-^2^ ail n by n orthogonal matrix 

and R . is k by k, righr triangu-lar, and non- 
singular. Q -denotes the "fir.st k columns of 0. 

'Let 2c(A ) denote the null space of A?* i.e.," 



and 



^0 ^'^^ 



^ (19) 



Xhe projector 'on* 2i(A'^ is ^ given by the matrix 

Further, suppose h lies \in the span of A, i^.» 

^ . • . 
- - ^ h - Aw -'-^ w a . , - - C2I) • 
i«l ^ 



are index- sets for those linear and 

non-linear functions, respectively ,. chat are'.in- 
the projecJtion. • . 

Step 2bS If ■LE(x^\e>r='<J. go to step 2d. • 
Otherwise let q'= Q«Q^e, [Recall that • Q. 



is the 



matrix made up o^f the last Tn+l-k) columns of Q]'* 
and j - {the i .that maximizes'. 

[iV LE(x^c> -:J^}. ' ; 



q^70^(i:,X^) 



whei 



re . A ^ [a^a^.-.aj^], then w is ^g^ven by' 



• • -1 T • 
^,w « R •^Q'-h. 



(22)" 



Write w * . I^^;^— -^j. 3 • "In the context of this 

jjaper, ;and irz the light o'f the introduction, we are 
• considering the columns of A to be a subset of 
^-cheA V<)^(z,x)'s where . i E(x,c). 

*v Tlnss the direction of .(5) is given by' ^ - ' 



• , ' IJqJI £l^70,(z,x^«| L 
T ^ 

..^ V4»_j(2,x ) > 0 go to step 2c, 'otherwise go 
to step 2d. - - ■ 

- ■ . • . »^ 

Step 2c^ Add a*column-to A and update Q ,and 



Q^Q^e. 



(23) 



accordingly . 
a = 7^j(z,x^ 



Writing A « [A:a]- 
we have 



where 



R' 



As we shall shortly see, we will 'actually 
build up Q by adding colunms to A one at a titae. 




I. T 
1. -^Q-^a. 



where Vj^ - Q^a, - 
-Now VfU-jaay -cake -a, produc c ot~Givens_ txansiormaticns,. 
C CO transform inco a vector having zero in 

V ^ 

all the compoBents save the first, .where* is of 
the form 



G - 

V 



0 



0 • 
-V 



'1 



where y and - Y occupy position v and v+1 
on the main diagonal. , Then ■ 



A- - [Q G 



- Q [r - Q Fr 



Set 



A - A. 

Q - Q.. 
R - 



-I ' . ^ , LE = lE\{j}. 

If -the cardinality* of' reaches n+1, go to 

Step 9 below. Otherwise go .'to step 2b. 

Step 2d ; S^ore and . In . and Q 

'respectively- 



2. 



. Store /R in R* 



[In the cases where LE(5c ,c) is originally empty 
Che scorage of Q_ and Q^^is unnecessary], 
k ^ 

>^tep 2e : If NE(xj ,z) go to step A. 



•T k 



Ocherwise sec 
j • {i chat maxi^iizes 

If * q'^Vo. (z,x^) i 0 go to step 4. Ocherwise* add 
7^.(2,x ) CO A and updace JQj^and R as ouc- 
liried- in step 2c above. 



i c N'E(x ,c) - k^}.. 



Sec - Kq u {j}, 
NE - NE\{j} . 



*Cf ct>e cardinality of u j^aches n+1 go co 

seep j^9/^cherwise ^''^ 

and return co cne'scart of seep 2e. 

Scep^3 : vDecermine che projeccion matrix (chis 
' seep involves inner iceracions) * 



Step 3a ; 



R- R. 



«• [Recall chac Q is. chAc V^rC *of the d composition 
that corresponds co jast those linear functions 

"""f ^ tx)~"~ inc luded -ln"Che -projec tion " dirririg-the 

previous iteration, ^fvstorage restrictions make 
it necessary^ Q ^noy be discarded and' rec ompu ted ] , 

Let q - "Q.,Qle. If LE(x^/c) - J^. - 4> , go tO 

."tiv"" k * 

step ..3d. OtJieVwi^e? choose j c LE(x ,c) -,Jq» 

add V<>j(2,xlV To-iivA and update Q and R as 

above. [Step 2c] . , 



Set 



\ LE - LE\{j):^. 

Xf q 3* 6' go to step 3b. [O \ " 

Step 3c : T5rop columns from A and update Q 
R ac cord ingly \ Suppose 



A - ^[9^»a 



2*"- "*^j-l*^j+l' 



^.e., A is^ A with the j th c6^umn deleted, 
resulting f ac'torization is of the form . 



and 



The 



where H is right Hessenberg and is obtained from 
R by deleting is jth c<?l*umn. The subdiagonal 
element's in H, which extend from the new jth 
- (previous (j+l)tH^ column of R to the last 
column can be removed by appJLying an appropriate 
sequence of Givens transformations, G^, Thus we 
may writ^ 



q = 0 
columi^s of 



Q[G^ ... 



implies that 
A, i.e., . 



e lies in the span of th 



Since A 



Aw « e. 

the vector w 



is given by 



-1 T 
w - R Q e, 

i.e., set y - Q^e ancf solve 



If w i 0, stop, X 
such that w. < 0- 



Rw - y. ^ ^ 

is optimal. Otherwise, choose 
[Typically one chooses '-j 



such that - w. is most ne^at^e]. 



and dropa the 



co^esponding a. from A, updating the Q and, R 



as outlined above 

. Sec J, 
■ til 



0 
LE 



LE + {j}. 



Step 3d ; As for step 

Step 3e : As for stef 
last statement is rep^ 
■start of step' 3e"] . 



ve. f?SH=&pt phafc the 
■ "return to /the • 



Seep. 4 ; Set 
Let lA - {1, 



d - - 

... ij} 



denote 



Step 5 ; Check optimum i3 reached , 
T<*ll *<Gstop and- Label >* 6 or if 



If 
stop 



c <cpstop. 



Step 6 : Linear Search 
« """"^"^^"^^""^ 

' The linear search is done directly on the min^ 

Imax function. Let d"" be the direction d of 

•step A with the first component deleted (remember * 

that the first component of d (d+) corresponds 

CO the z and d corresponds to x c r")^ Then 

d has 'the property chat it wil3c ti^ to decrease 

the subset of the active functions at the point * 

fi^(x); ^i^t^)- ••••^^i (i*e., those we put ' 

^ into Che projection) by the same amount. (Ihus 
decreasing M^(x) , since , ^by construction, the 
remaining active functions at the point/ will 
locally decrease along d"^ , as this lis the basis 
on which the projection matr<U was determined ) .Thus 
Ic 

at the point x we move downhill along the valley ' 
defined by the minimax functions f (x) f 

i^ • ±^U)^ 

---•f^^(x). Note that if we -consider only 'one 

, function in P, say the function f (x) , then ' - 
- -1 . p 

d - - 7f (x) , which Is the sTeepest descent 

direction for the function f (x) . , 

, ^1 . ^ ' , . 

Determine Tv> 0 such that ^ 

Max -f^Cx^ + Td"), i e [M3 J" 



is minimized. Call this x, t^^^ 

For details of how this is done and whether 
exact minimiiatjion is re^uir'ed, see the' sub- * 
■ algorithm ":^e Linear Search Algorithm" below. Put 

- ' k' * k • . 
^* .x-^x +T d.' 

opt 

T ... 

^tep 7f Decision a« to whether to do the vertical 
step or Qot^ The vertical direction, amounts to 
attemjJting to make the near active functions 
exactly e<iual, and by doing' so an eStort is made to 
' get exactly on the line of .the discontinuous ' 

derivatives, whic'h Is very, desirable when we are 
clcJse to the solution. We expect that we will be 
jr close to the solution either when the active 

functions remai^ the same at each iteration or If we 
are trying to consider more than n functions in 
the^projeccloir. ^ ['Actually,, even if we .are not near • 
' the. solution, should either of these two phenomena 
^ occur we might -need to "reach the valley f loof " in 

order to 'move away from the current situation]. 
- AlgorithmlcaUy, if the number of .active functions 
- have not changed in three consecutive iterations, 
K 0, and lldjl <.l, or if Label - 6 (see" step > 
9J go to^step 8. Otherwise, set 

k+1* k ' , • ^ 



k ^ k+l 



and go to step 1. 



i£«2_i: "The Vertical 'Seep [involves inner 
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iceracions]. 

Set - M^(x ). . Determine E(x ,g). (Note 

that X '"is "che poin ^^bbtairfed ■ -~ 

' horizontal step). 

Stepjla; -Set - , " $2 - R " R- 

Ste£_8b: As for step 2e above^ [except that c'he * 
» .last statement is replaced by ""return to the 

start of step 8b" and "go to step 4" is replaced by 
' "go to step 8c"] . ' 

Step 8c : v(x^,H - - Q^r""^C - 
* where ,^ - (<j)^ . . . , / , 

^temp ""^^ + V- (wlt'h 1st component missing) 
k - 

- X + V . ^ 



If 



Set vs 



1 



Max f ^ (x ) 



= Max f ^ (x*") , 
i^M ^ 



1. 



k+i 



X , k 
temp 



k+1. 



Lab<el - 0 ^nd go to step, 1. Otherwise set vs - 0, 

k+1 k 
,x - X , 

■and go to step 1. 



k+1 



is the 



<Note that f (x^) + Vf^ (x^v' - M^Cx^f^ v"^ + 

where jc^l < c, ^ - 1, 2 , . . . , j , and v"* 

firstf component of v(x ,c). Therefore the 
. linearized active minimax functions at x^ » will be 

within ' c at the point x - + v*V 

temp ' 

Step_9: Trying to put " (n+1) constraints in the 
projection, i.e., ^ 

1^0 '^ KqI - n+l. 

(•^0 ^0' " means onl of two possibilities, 

• either a) we have reached the neighbourhood of the 
optimum, or b) we have constraints considered 
active that actually are not. 'This situation is 
handled in two ways/ Firstly, Iby ensuring that- we 
take a vertical step and;secon%, by reducing e. 
Algorithmicallyj set label - c - c/10 and go 

to step 8. /r - ^ 

The Linear Searcli Algorithm [ 

Ste2_0: If vertical steg was successful <i.e.» if 
vs - 1), go to step 4.*^ 

Ste£_l: Estimate any new functio'n to become active. 
We consider ali inactive cons^aints (> and 
estimatre, in turn, the ^tepsize to make each 6 ■ 
zero. Hence we calculate ** ^ * 

• J € I(x , e). ' (24) 
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In other vords , if was linear we calculate S 

so that ❖,((2, .x^) + T.d) - 0 and in general we. 

J J It 

Tineari'2c""the~(i^^s ~ Xet r (x ) M~(x )— f or-aoine 

m» ^j^^^ inactive function at the point x 

and . as calculated by 24. Then we want the 

linear approximation of the functions* f and f • 

about the point x to be ir^tersected at the 

^ ^ k * — 
point X + Tjd . By construction q is 

orthogonal to V(z - f (x)) at the point x , 
i.e., 

Vf (x )^d 
Also, by construction of 



(25) 




By using the fact that z^^ J^£^^ ^ ^^^^ 



From 25 



f'j(x^) + T^Vf^CxS d" 



f (X^ + T,Vf (X^)'^d" 

m J ™ 



M^CxS + T^d"*" 

f,<xS + T^d^ 
f (X^ + T 



(26) 
(27) 



Comparing (26) and (27) we can see that we have 
the desired result- ^ 

Step 2 : Omittiag unlikely values of ♦ estimate 

the optimum, t , by'linearizing the minimax 

°P^k • k 

function about x . For j c t(x ,c) do the ' 

following*. If "^j '^j^'^max ^^S^^ct it as 

inadmissible. Otherwise calculate 

k.T - 



^ij ^i^^ ^ '^j^fi(x")'d , i € [M] 



where 



Put 



Vf^(x) 



Now, determihe 
S 



-—f. (x) , . . . , -r^- f . (x) 
3x^ i 3x^ 1 

. - max ^ _ . 

I such that. 
F 



Mln f , 



j e I(x ,e)\{jlT^* <0 or t'^> Tmax}. 



But 



opt a 

Step 3; - Determine if t 



opt 



is acceptable* 



Calculate the true minimax value at 



Ak 

X 



opt 



If this new value is an improvem^t over the old' 

Otherwise go to step 4. 



■ k Ak 

value, set x » x 



Step 4 : Use cubic line search on the maximum of 
the functions taking ^ as an upper bound, to "'^ 

.obtain.- tha-new„_.x ^-If. i ^_ _ is^availablcirom 

. t opt 

stejp 3 [i.e. , if vs - 0] . 

2.3 Some remarks on the alRorithm * 

i) ' As id noted in [4] the QR decomposition and 
its update when a column is added or deleted from 
A, arises with such frequency that a wealth of 
techniques have been developed to handle the 
situation. The method set out here, corresponds to 
that in [11].. It is used presently solely because 
it is the simplest to explain. For alternative 
methods, more detail^, and references, see [4]. 

ii) In practise we do not use an exact cubic 
linear search bgt merely ask for sufficient 
improvement in the minlm^x value. 

ill) If in step 7 of the main algorithm we decided 
to do the vertical step and it was successful, we 



dispense wiph the estimation of^T 



opt 



as above and 



merely do the cubic search . The motivation, for this 
is as follows. The estimates for are based on 

the surmise that some new function will become 
active whereas the vertiqal step is based on the 
assumption that this will not be the case. 

iv) In the above algorithm we have assumed^ that 
the cojlumns given by V(t)^(x,c), 1 c LE(x,c) ^e 

linearly independent. The situation in which the 
qg^lumns are dependent corresponds precisely to ^ 
degeneracy in linear programming. As was done in 
[4], we will regard degeneracy as a condition > 
which can be cleared up through the use^ol 
negligible perturbations, of the data, and wC^'shall 
ignore the linear-ly-dependeiit case. 

v) Under some fairly natural assump^ons the 
above algorithm can be proved to be convergent* ' 
The "proof s are somewhat lengthy and are given 
elsewhere [5]. In the case of solely linear 
functions, finite convergence can be proved. 

[See [4]]'. , . 

2.4 Relationship" of the above with other 
algorithms 

I) The algorithm differs from that given in [5] . 
in that it is a stable implementation. Furthermore, 
whenever possible, advantage is taken of an^t linear 
functions that might be active.^ This is done by 
preferentially entering the linear functions info * 
the projection and storing that part of the 
decbmposltion (QR) that corresponds to the linear 
functions alone. Thus, instead of recomputi-ng the 
entire decomposition at ^ch iteration, the linear 
part can be updated* 

II) • As was remarked above, in the purely linear 
case, the algorithm 2.2 does nat^ quite, reduce to 
the algorithm of [4]. • This is because la that 
algorithm M_(x) « Max|f (x)j and, in addition, 

i 

the linear search differs from.t^a-t above. Thus^ 
the corresponding P* is ^ 



Min z 



^subject to 



2..- |f^(x)| S 0, 



1 £ 



(28) 
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to 



is apparent that the above problem is equlvaionC 



, Min z 



.with corres 



z - f (x) 0,) 

ponding ' ' 

Mln Max (f. (x), - f,(x)} * P* 
' X i£[M] ^ • ^ . 




If one formulates the problem this way,' elimifiates 
the vertical step on account of it being super- 
fluous ^n the purely linear case, then, excepting.' 
the linear search,— one has, essentially, ti^e * ' 
algorithm of [3]- For completeness I point out 
that they use fast Givens without square • in 
their decomposition of A instead of th •5, 
given above. (The elements of fast Givtr . 
given in [13] ) . 

There are three linear searches mentioned in 
[3]. besides the complexity introdtK;^ed by the 
absolute^ modulus function, none of the methods 
correspond exactly to that given in algorithm 2.2, 
(henceforth referred to as LSA(2.2)). For 
simplicity we will iiescribe equivalent forms for" 
the original problem P. 

Linear 'Search. Algorithm A . (LSA(A))- 

Ad for LSA(2.2) but determine I such that 

*\ ^ k 

T IS the smallest positive: T , j e I^x ^c), 
^ ' ^. J 

[i.e., locate jche first breajc-point reached in the 
direction d .of^the pifecewise lineax function 
on leaving x ] . 

Linear Search Algorithm B , (LSA(B) 
Step i : As for step 1 of LSA(2.2)t 



^^ep 2 : Put 
j c I(x^.e); 



T as the smallest positive t , , 

. J 



Step 3 : [Determine^whether is still decreasing 

in the direction- d"] . 
- * k 

If d ^fjj^C^c )> 0, terminate the^linear search 

algorithms with t ■* t,. 

opt Ji' 

• Otherwise it is necessary to make the 
follov/ing redefinitions ' > " 

X - -Ij^T^d 
.Z^ -^^(X^ + T^d") ' 

and return to step 1. . 

Linear Search Algorithm C . LSA(C) ^ 

This algorithm wsT^ a mixture. of LSA(A) a^nd (B) . 
Specifically^' it used LSA(A) until such plme as 
q - 0 in step 3b of algorithm 2.2» above. Then in 
step 6 it. used LSA(B) . In subsequent iterations 
one returned to- LSA(^) for seep 6* until q - 0 
again in step ''3b, and so on. 

Schematically we have the following 
situation for the linear search." 




Figure 3 . 

a corresponds ^to t f t-om LSA(A) 
opt. 

^ S " " " LSA(2.2) 

Y " " • " LSAr(B) 

Using LSA(A) , with the exception oT , possibly, 
the^ f irsc n+1 iterations, each^ of The. points 
k 

X wet^e vertices for the (linear programming) 
Dj/oblem P*' (or at least satisfied a maximal 
^mber of active constraints*' in case the problem 
was rank deficient). It is ior this reason tHat 
they preferred strategy LSAWV). However, in the 
case where the f^(xy's are n^n-linear , 

essentially, as is usual io npn-llnear programming, 
one is looking only for a sufficient deci;ease in 
the objective function, M (x). .Strategy LSA(B) is 
undesirable in li.cil it ^requires coo much 
additional corpur.iri. . ' ^ 

iii) The alRorithn o' . :0] corresponds exactly to 
the algoritJm ..hov. : -ing LSA(A) iexcept that 
eiine is restricted to Haar-condition problems) 
starting at a vertex to the (liheSir progr^lraming) 
.problem P'. In ^Hdition to being able to handle 
non-Haar problo:::. . the treatment given above* is 
more flexible. T;. particular the addition of 
constraints can : j handled in ^ natural way that is 
particularly simple in the case of linearly 
COT L:-.,ined problems. 

iv, ?'^rh^::s the best known, algorithm for handling 
problei!is 01 ty;?e (28) iu the linear case is that of 
Barrod.^ - a-d Fhillips [2]. They, in fact, add an 
additior-^: lir.ear constraint, viz. 

^ z > 0' • 

to ?*• and then solve the ciual linear progra^m. 
The structure- of the problem enables them- to 
condense the resulting tableau by supressing all 
but m of the columns. There is some suggestion 
that i:; practice the dual form, at least for 
linear probletss, is the best approach to the, I 

problem. However, to jche best of the pre/ent , 
author's knowledge, th^ere Is, as yet, no well- 
documented evidence to support this assertion- In 
particular , ^let us formulate the algorithm of 
Barrodale and -Phillips in terms of, the primal. The 
algorithm consists of three stages*. -More 
«^eci£ically, suppose the problem ^nder 
'onsideration is . , > 

... . "J'"""^' 

and the rank of A is k. ' " 

In stage 1 one chooses the residuair of 'maximum 
absolute value ^nd essentially, uses the steepest 



V ^ ^^^|^||b-Ax| 



- Max |b 
liiim ^ 
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V. . - 

dcscenc di-recclon to bring its value co^zc-ro. Ont 
then updates the residual values chooHinj; that 
which Is now nuixiiruil in absoluCc value and 

- b r i n g t OK i t -d own -ta Zi^r A ^- whiiii t - ii till -.holding 

Chose residuals Chat are already at zero zero. 
One Chen iterates until k. reHiduals. are at zero. 

This iTs readily accomprlished by means 
afuiloROus CO those above for algorithm 2, J. For 
example, suppose ^ , 



0 1 



1.: 



J-i 



and 



Define 
where a . » 



Is maximal . 



r rj j' 

? as in (3) with .N - [a^ 



c > 
r 



The required descent direction is given by 
Pg.^here g * -^^ sign(e^). . (sign (c^) " 1 

0, and if e < 0 and 0 otherwise); 



if 



Since in stage 1, ^typically one increases the 
number of residuals with value zer\ by 1, the 
orthogonal decompositions, plus corresponding up- 
dates, are applicable here. Stage 1 is . 
terminated when k . residuals haj/e the value ^ero. 

Stage 2 consists of increasing all the 'k 
residuals (in absolute value) s Imula teously , until 
k^l residuals have the same absolute value. 
Stage 2 involves only one iteration. 

Stage 3 'is equivalent to the wel'l-knowji 
exchange c^lgorithm for linear minimax approximation 
(see for example [.lA]). ^ 

Aga<in Stages 2 and. 3 caji be accomplished with 
linear algeljra analogous to that of algorithm 2.2, 

The above is only a very brief outline of Che 
algorithm of ,Bafrodale and Phillips. T^ie de calls 
of Che condensed simplex implemenCacioi\ are given 
in [11.. ■ ■ ■ ^' ' 

In Che llghc of che above. a6ramencs one is able 
CO see that che "dual*' formulac;[:on of Barrodale and 
Phillips can be realized ii\ a scable way by a dir- 
ect fortnulacion as for •al gorichm 2.2. Ic .chen seems 
nacural co ask che quesci©n chat if t^e 5ual approach 
' to the minimax problem is superior to Che direcc 
approach , .when is ^ dual approach indirect? Turther- 
more, Barrodale and Phillips claim that the maiir^ 
feature of their algoritHm is that ic t>nters stage 
3 with a value close to optimal. However, if one 
compares how .ftne arrives, at stage 3, in cerms of 
che primal problem, co che analogous sicuacion in 
algorichm 2.2 [i.e., vhen one reaches a vertex (or- 
poinc where q » 0, more generally)] che apparenc 
sruperiority of che *^indirecc" or '*dual" mechods 
iappear^ surprising. 

v) Much of ^che ideas of . [Il7] are in face 
equivalenc co<chac of che horizontal direction of 
aigprithm 2.2^ However, the vert^al^direction is 
dispensed with ac che cost of havin>i to reduce the 
tolerance LP which functions are considered active, 
to zero.. This makes ultimate convergence difficult 
(cf. [7]^ and [9] for'a similar resilt). Further- 
more the numerical .aspects of the Zangwill 
algorichm af^ noc considered by hitn. In parcic-- , 
ular, no linear search suitable for minimizing 

M^(x) is givea. Finally, Zangwill introduced 

f / ^ 

separate cases .dependent on whether a certain 
matrix is of full, -or almost. full, ranks. Such a 



-^^f^(7c) 



differentiation of cases is numvric al ly ' 
undesirable und does not occur in algorithm 2.2, 

_ 2^3 Some' additional remarks and conclusions. ^...^ . ...... 

As is noted abov«, and^n [4] for the linear 
case, the addition of consit raints to problen/ P 
can be handled vin a natural way. This is an 
'Mmmediati? consequence o'f the fact that algorithm 
2.2 is based on\pi*oJeCtions . 

One might Jilso rem;irk that since the 
projections in algorithm 2,2- arc orthogonal, at 
least in terms .of the non-li.ncar programming 
problem P* , the iterations consist of stcepcst- 
desccnt-like iterations in certain subspaces. It 
might be suggested 'that one might be able to 
■determine ' Newton or Quasi-Newton type iterations by 
using nonrorthogonal type pru^jectiSns ^see for 
ejcample [15] and [12) for parallels in non-linear 
programming) . However, since the objective- 
function (z) in P* is linear, the s traight forward 
non-orihpgonal projection will not do £he job. 
However^ ^consider the case of just one function 
f^(x), say, active. Algorithm 2^2 gives 

for d - i.e., stoepeiit descent, however, for 
general non-linear f the corresponding Newton 

direction - G^^(x)f(x) is well defined [where. 

denotes the Hessian of f ; here assumed invertible]. 
In other words, a' second*-6rder type generalization 
is not obvious, even assuming it is desirable. 

The above is meant to be a sumniary of one 
(direct) approach to minimax problems via 
projections. Some attempt has been made to' relate 
the approach to other algorithms designed' for the - 
same problem, It^ is aliio hoped that the results^ • 
reported here indicate that there is still work to 

done in the area with sugges"tions as to some 
ayenues of 'future research. 

Numerical results are not reported here.*^ 
Suffice it to s;ly Chat numerical results have oeen 
carried out in the context df applications, general 
non-linear and linear problems. They are reported 
■ in detail in rAerences [5] , ' [3] and [6 ] . • 
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Abstract^ 

^This pfeper dlseusses two algorithms for non- 
linearly constrained optimization. These algo- 
rithms — the penalty and* barrier trajectory algo- 
rithms — are based on an examination of the tra- 
factories of approach to the solution tfiat charac- 
terize* the quadratic" penalty function and the 
'logafit"hnxic barrier' function, respectively. Al- 
though closely related in principle/ tffe two algo- 
fithms- display important differences in their 
implementation as 'well as in the properties of the 
generated iterates* The discussion iJill emphasize 
the^numerical aspects of implementation of the 
trajectory algorithms, with particular attention 
to the. choice of reliable methods for carrying 
out tjie required computations. 



, Intro duct ion . 



:> The pr^lem to be considered in this paper 
is the foi lowing: * 



PI: 



fiinibize 



P(x), 



^n ^ 



_ 

subject to c.(x)>o, i " l,2,*,.,m» 

where F(x) ^d {c. (x)} are prescribed non- 

,1.6 . 

linear functions. The function F(x) is usually ' 
termed' the "objective function", and the set ' ' ' 
{c^(x)} t^ set of "constraint functions". It 

will be assumed for simplicity that and "f^j^^J 
' are twice continuously dif ferentiable, although 
the methods to be discussed will cope wich occa- 
sional discontinuities. 

The solution of PI will be denoted by ^x*. 
In all problems to be considered, the first- and 
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second-order Kuha-Tucker conditions are assumed to 
be satisfied *at x*, so that there exists a vector 
X* of non-negative Lagrangla multipliers, corre- 
sponding to the active constraints at x*, satisfy- 
ing - ' " . 



g(x*) - A(x*)A* =" 0 



(1) 



where g(-) is the gradient of F, and the columns 
of A(*) are the gradients of the constraints 
active at x*. 

It is customary to state problem PI with two 
kinds of constraints — equality constraints (of 
the form c^ (x) * 0) , as well as the inequalities 
given above in PI. This distinctibnrwill not be 
made during the present discussion, in order to 
avoid the introduction of additional notation. . The 
treatment of inequality constraints is typically 
more complicated than that of equality constraints, 
and the algorithms to be discussed can deal in an 
obvious way with equality constraints. 

because the fJroblem PI can not, in general, 
be solved explicitly, a popular approach- during 
the past decade has been to transform PI into a 
sequence of unconstrained minimization problems. 
The most common, such transformation has fieen 
\eff ected by the xise of penalty and barrier function 
'methods, which are discussed at length in, for 
example, *Fiacco and McCormick (1968) and Ryan »' 
(1974). This paper w^4l not review these methods, 
but>f heir properties are" critical in the deriva- 
tion Df -the algorithms to be discussed. 

The quadratic} penalty function corresponding 
to the probleip PI, is defined by- i 



P(x.p) S'F(x) + f 



i€l 



(c^-(x))^ 



(2a) 



194 



20€ 



V 



whcr^^ . is a Subset &1 the indices {£i^v,.,m} 
(usually > Che sec of conscraincs whose values are 
''less. Chan a small' pis icive nuober), and p is a 
vposicive scalar Cermed Che "pedalcy parameCer". 
M-T;^e quadratic penalcy funccion m&y also be wriccen 



as: 



P(x.r) = F(x) + ^ r"^ I- (c,(x))^ . (2b) 

• where r - 1/p, Ler x*(r) denote an unconscrained 
^ t minimtim of P(3;»r). 

The logarithmic barrier function corresponding 
to the problem PI. is given by: 



B(x,r) =.F(x) - r ^ B'nCc (x)) , 
i«l ^ 



(3) 



where" r. is a positive scalar termed the "barrier 
parameter". The logarithmic barrier functi^ is 
defined only at points that strictly satisfy all 
of the problem constraints Let ^(r) denote 
an unconstrained minimum of 5(x',r} 

Onder certain mild conditions, there exists 
*r > 0 such that for r < r, 3^^) ^^^^ 
are continuous functions of r^ and: 

lim x*(r) »'x* ; 
• . r*- 0 ^ 

lim x*{r) x* • 
. r - 0 ^ 

- ^ 

Although penalty and barrier function methods 
display several ^good features, they suffer^^^om . 
certain theoretical ^and numerical 3efects -—.in 
particular, both require the solution of a 
theoretically infinite sequence of unconstrained 
problems. Furthermore, in practice each suc- 
cessive unconstrained sub-problem is more dif- 
ficult to solve, because the Hessian matrices of 
the pena'lty and barrier /functions become increas- 
ingly ill-conditioned as r approaches zero, and 
are singular in the limit <see Murray, 1969a) . 

The continuous' lines of minima in 
diefined by x*(^r) . and x*(r) are tenned the 
"penalty trajectory" and "barrier trajectory'* of 
approach to x*, respectively* "The analysis of 
these trajectories, and a detailed description of 
the trajectory algorithms, have been given else- 
where (Murray, 1969a, b; Wright, 1976; Murray and 
Wright, 1976b)- for the purposes of the present 



discussion, only a'brief* description of the 
underlying motivation will be Stated, 

The trajectory algorithms are based on -tising • 
the properties of the trajectories to generate*a. 
sequence of iterates that lie. in a' neighborhood »' 
of'the appropriate trajectory, in order to^-pdmic 
the a|>proach to x*V of the iterates from a penalty 
or Sarrier function .method. • B.ecause it is possible 
to characterize a step toward the penalty or barrier 
trajectory without assuming that the current iterate 
is in a close neighborhood of x*, the derivation 
of the trajectory methods does not display a ^ . 
stringent^ dependence on properties that hold only 
in such a neighborhoods Moreover, it is not neces- 
sary for any of the iterates to lie exactly on the 
trajectory (as.^ in a penalty or barrier function ^ 
method) . : ^ _ 

- • At each iteration of a trajectory method, the 
search direction is computed a^step toward some 
point on the desired trajectory. The partitrular 
point to be aimed for depends on the current value 
of the penalty or barrier parameter. The solution 
X* is also on the trajectories, and the 'target 
point will ultimately become arbitrarily close to 
X* as the algorithms cpnverge. The penalty or • 
barrier, parameter may be adjusted at each iteration 
of the trajectory methods; however, the choice of ' 
the, parameter value .is not critical, since a step 
to a neighborhood of a point on the trajectory 
corresponding to r is also in. the neighborhood/ 
of a point corresponding to (l+€)*r, where e is 
small. The numerical procedure for determining 
the* search direction in both atgqrithms ' is well- 
posed, and the approach to the limit of the penalty 
or barrier parameter does not cause any ill-condi- 
t.ioning, - ■ , • 

This paper will emphasize some numerical 
aspects of implementation of the trajectory methods.^.v 
with particular attention to the choice of reliable 
procedures for carrying out -the required computa- v 
tions* The emphasis on the details of implementa- 
tion is deliberate; even within an algorithm ^^C 
has been designed from the ouCsec j:o h€ robust;-, . 
.additional safeguards are necessary co pfocecc 
against failure or illogical resulcs when the under- 
Lying assumptions are not satisfied. ' 
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2. Description o'f Trajectory Algorithn^ 

T. Only certain key aspects' of impleaentation of * 
the trajectory, method^ have Wen selected for dis- 
cusslon^iui Secj;don 3. . Accordingly, the descrip- 
*t ions given' here of ; the p^alty and barrier trajec- 
tory algoritjims are slightly abbreviated, and do 
not contain, all the computational details. A 
niRlete description of both algorithms is given 
n Mui>ray and Wright (19 76b).. 



for the quadratic penalty function; the step to 
b^ taken along the search direction is then chosen 
^t<x achieve an accept^le decrease.^in^ the penalty 
function- 

• ,' ■ '■■ ' 

2.1.2 ; Calculation- of the searfch direct ioji- 



^ At the beginning of the k-th iteration of the 
penalt)^ trajectory algorithm, the followlrig vectors 
and matrices are assumed to be .available: . 



2-4. P. 
2.i^lV 



Penalty Trajectory Algorithm '^ 



2 • 1^. rS-xP roperties of the search direction 

/ ^For the penalty trajectory algorithm, at 
each iteration the search direction, p, is 
(ideally) constructed as the solution of the follow- 
ing quadratic program: 



QPl: 



IT T 
min Y p Sp + p g 



subject to A 



-c - - ?i 
P 



Xk), 



- X i ^ an approximation to x*; ^ 

(k) \ 
evaluated at x ; \ 



the vectoi* of values of 



- g^"'» the gradient vector of F(x) 



- A 



evaluated at x 
, the matrix whose colu^iins are the 
gradients .of {c. (x)} evalxiated 
, at x^"^); . 

- S ^, an approximation to the Hessian 

matrix of the Lagrangian function 
at x^'^^. ' 



where c denotes the vector of constraints cur- 
rently considered -"active"; A is a matrix whose 
columns are the gradients of the active constraints; 
A is an ^estimate of the Lagrange multiplier vector; 
p is the current value of the penalty parameter;, 
g is the gradient of F;',-and S is a matrix that 
approximates the Hessian of the Lagrangian function. 

Let y be a matrix whose columns form an 
orthogonal basis fdr the range of the columns of A, 
•and let 2 be a matrix whose columns fotm. an 
orthogonal -basis for the corresponding null space, 
i.e.-; 5., 

A 2 - 0 , . 



^ 'The- procedures followed du^ng the k-th 
iteration to Compute the next iterate are; 

(1) ' An "active set" of cdtistraints* is deter- 
mined, containing n elements (see Sect^^on 3.1 
for a discussion of the case where the active set 
contains more than n elements) * The vector of 
active constraint 'values will be denoted by c, 
and the matrix whose columns are the gradients of 
those constraints will be denoted by A. 

(2) Fa^torize A such that 



•QA 



T 

Q Q 



I . 



T 

2^2 - I . 



If A* has full Column rank n) , and if 
T 

. the matrix 2 S2 is positive definite, .then the 
"so^l.ution of QPl, p*, can be uniquely expressed as 
£he sum of tw3 o.rthogonal components: 



where R is an upper^^triangular matrix. Define 
the matrices' Y and Z hy- partitioning Q as: 



For sufficiently large p, the. search direction p* 
, ^so constructed will always be a descent direction . 
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(3) Determine an estimate, ^, of the Lagrange 

multiplier vector- If A has full solumn rank, 

X is the least-squares solution of minBAX-g^'^^ I^, * 

.and is given by the solution of the triangular 



I 

20S 



system: 



If A is ^ank-def iviient, the vector X will be 
t^en as the minimuta-length least-squares solution; 
it is obtained by extending the factorization of 
Step (2) to: 



QAV - 



D5^B <.MQp^l* ■ and;-. 5p^0 <nlp^t , /-(o) 



where R is a tion-singular upper triangular 
'matrix and V is an orthogob^Cl matrix (see » 
Peters and Wilkinson, 1970, for further detail's)^ 

(4) Determine an appropriate value* of the 
penalty parameter, p . (Murray and Wright, * 1976b) 



(5) Compute the'vector 



Pj^, as follows.^ 



If- 



A has full rank, p^^ is obtained by solving the 
linear system: 



for M '^a reasonably large positive number (say, 
1,000)*. 

(a) If the test (6) is satisfied (as i-t ' ' 
almost always is in practice), comp.ute ■■ 
solving: 



LDL^ - -2^(g^'''^ + S^^' Ypj^) ; 



then form the search direction as: 



(b) If the test (6), is not satisfied) the 
two orthogonal portions of the search 4ii^€'ction 
are not' well-scaled, and<^he .following re-scaling- 
procedure is used to.Mjust for the imbalance- 

if Op E' > MOp i, ^e£ine a scaling factor 8^ 
as - 



*T ^ -J * . 1 

Ap-A YPj^. R Pj^-c-^-X 



(5)' 



In this way, the direction Yp^ saris fies the 
linear equality constraints of QPl. If A "is 
rank-deficient, p is a least-squares solution 
of (5) , computed using the complete orthogonal 
factorlzatioi^^{4) ; the linear , constraints of QPl 
will then "not be exactly satisfied. 

(6) Determine the modified Cholesky factor- 
ization of the matrix S^^^Z.. ^^th this pro- ■ 

. ' ^ > T (k) 
cedure, the mansrix-^ S Z is augmented (if . 

necessary) by a positive diagonal matr^.x, E, 

chosen to make 

ically) positive definite. Let LBL 
computed factorization, so that 



LDL^ « Z^ S^^^Z + E 



and let 



(Z^S^.^.'z + E) 



strictly (numer- 
be the-. 



Z^ S^^^Z 



is. 



where E'. 'is: identically zero rif 
■sufficieritl'y positive definite (Gill and Murray, . 
1972a)*;! .\- ■ ■ . ■ - ■ 

ill) Determine the^ectir p^^ by* solving 



p = Ypj^+ Zp^ ; 



otherwise, de^ne a scaling factor $2 



and let 



p - B2 Yp^ +;.2p^ 



Test Aether: ■ ' 



(8) Determine a step length, a, that .generates 
an acceptable reduction in the penalty function 
. F(x, p) , using a safeguarded cubic or parabolic 
step length algorithm (e.g., the procedure de- 
scribed In Gill and Murray^ 1974) . Special care 
must be exercised in the step length algorithm to 
avoid diff lculties/±£. P(x,.p)« is unbounded 
below along t^e glK^en search direction (see 
Section 3-5.1). 

; (9) Set x^^-^^ to x^^^ + ctp, and" return 
to Step (1) . 



.2.2; * Barrier Trajectory Algorithm 



2.2.1; 



Properties ^of fee search direction 



The search direction of the barrier trajectory 
algorithm 1? (ideally)^ constructed as the solution 
of the following quadratic program: 



PE2: 



IT T 
min J p Sp + .p g 



subject to A p 



irfiere d^ 



r/A^; c denotes the vector of 



i 1 

constraints currently ccmsidered .'^ctive"; A. is 
a maprlx whose columns are the gradients of the 
active constraints; l is an estimate of the ^ 
Lagrange multipliers corresponding to the active 
constraint!; r is the current value of the barrier 
parameter; g is the gradient of F;^and S is 
an apBfi^xima^ion to the Hessian of the^ Lagrangian 
function. " . ^ 

' It is essential "to achieve a reduction in the i 
barrier function B{x,r) at each iteration, ' ^ 

because the barrier function serves as a conven- 
ient "merit fianction" measuring progress toward 
X*. The derivation of the barrier trajectory algo- 
rithm indicates that the search direction given by 
the solution of QP2 may not always be a descent 
direction for B(x,r) • Therefore, the null-space 
component of the search direction miy alternatively 
be computed to minimize a quadratic approximation 
to the Lagrangian function, independent of the J 
component in the range of the columns of A. This 
.alternative formulation of theosesrch direction is 
necessary because of the quite different roles of 
"the penalty and barrier parameters as the solution 
is approached. 

2.2^2. - Calculation of the search direction 

At the beginning of the k-th iteration of the 
.barrier trajectory algorithm, the same vectors • 
and matrices are available as for ^he penalty tra- 
jectory algorithm* The Iterates generated by the 
barrier trajectory algorithm necessarily lie in 
.the strict interior of *tHe feasible region; this 
■ algorithm is intended for use on problems where 
some or all of the problem .functions may be illr 
defined or undefined outs'ide the feasible region. 

The computational procedures followed during^ 
the k-th iteration are: . . ^ 



(1) ^Determine the set- of" "active" constraints, 
denoted i&>*c ^ (see Section '3.2) ; form the matrix 
. A, whose columns are the columns 'of A^*^ corre- ^ 
spending to the active set< By construction, AT ^"^^ 
has <^ n columns . ' " ' / 

(2^, F^ctprizc A such that - * - 



QA 



Q Q - 1., 



as before. 

(3) Determine the Lagrange multiplier esti- 
mate X by solving: ^ 



so that X is the least-squares solution of 
mlnDAA-g^^^ I^J ^ similar procedure to that given 
for the penalty trajectory algorithm is followed 
if A is rank-deficient . If one or more compon- 
ents of X are negative, the constraint corre- 
sponding to the most negative is deleted from the 
active ^et; the modified A is then factorized, 
and the new X vector calculated for the re- ^ 
defined active set. Since the new A is simply, 
the previous A' with one column deleted, the new • 
factorization caix be^ obtained by a simple updating 
scheme (Gill, Golub, Murray, and Saunders, 1974). 

(4) Determine the barrier parameter, r 
(Murray and* Wright, ''1976b) . 

(5) Determine'the vector d according to 
the following rules^ Define s «^ Ocl'+ Ilz"^g^^^3, 
and set e = y r/s, where y > 1 (say, 2). Let 
r be the barrier parameter from the previous 
iteration; then: ' - - 

if X^ > e,'set d^ « -c^ X * ~ 



if 



X^ <_ -e, set. 



otherwise. 



1- 



-(1 -4-) c,; 

.r ^ 



(6)' Compute the vector p', ■'which is the 
solution of the linear system: 
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2i0 



in this v^y,. satisfies ^he^esired linear' 
. ^equality, constraints of^ QP2- f or.\bo6e probl^ ' '* 
cops traints for whic^^ X^ :^.suf f iciently .P9sitive;' 
an alternative relatidhship is satisfied fo.r each 
"active." constraint that has an insufficiently . 
positive*" niultiplier estimate. Again, the rank- 
deficient ■ case is created as for the penalty' tra- ^ 
j^ctory algorithm. ^ ^ ' 

(7) Compute the modified Cholesky factoriza- . 

, • T (k^ f 
tion of Z S 'Z (as in the penalty trajectory 

algorithm) ; the factorization will be denote^ by 



T ■ 
LDL . 



(8) Determine p„ by solving: 

N 



LDL*^ p^ 



Test whether: 



-2^ g^^> 



flp^il < Mfp_,ll and lIp_B < Mlp.D , (7) 

for M * a reasonably large« positive number. 

(a)- If the test (7) is ''satisfied, obtain 
Pjj by solving 



LDL^ p^ « -Z^Cg^^^ + S^^^ Yp^) , 



and define the trial search direction as: 

■ ;t - ' ■ 
If p is not a descent direction for BCx>r), 
\re-define p as: . 

, •/ , ■ 

■ 'Vj - 

This latter definition is guaranteed! to yield 

a descent direction for BCx,r). 

(b) If the. test (7) is not satisfied, then 
adjust, the scaling, as in the'pei^^lty trajectory 
algorithm. . 

, C9) Determine a step length, a, that 
accomplishes a Citable reduction in B(x,r), 
u£;ing special procedures designed for one-dimen- 
sional 'minimization' with respect to the -logarithmic 
^Jaarrier function '(see' Section 3.5.2} .^ During the 
search procedure, record whether violation of any ^ . 
constraint currently considered "inactive" restricts 
the step length; if so, this constraint will be ^ 



added to the active sec .at the next iteration 
^to Step' (1) 



(11) Set X . to^ X ' + ap,>and return 



' 3: " Some Considerations of IJumerical Analysis in ^ • 

Implementation of the Trajectory Algorithms, 
; In this ^ecjfion, .we consider some selected » 

^aspects of rmpTementation (of the trajectory methods, 
from, the viewpoints of nume^^al analysis and ^ 
algorithm definition. will- be stressed through* 
out . this discussion that an implementation could 
not achieve practical success if the definition of 
the algorithm' depended critically*on properties 
that hold only in a close neighbqrhood of x*; the 
algorithm should produce sensible results, even 
' when such conditions are not satisfied at the 
current .point- 

3.1. Selection of the Active Set for the Penalty 

■ 

Trajectory Algorithm 
The"* "active- 54t".. of constraints is defined 
at each.iteration^ of the penalty trajectory algo- 
rithm as the set* of constraints whose values are 
less than*» specified 'small positive number. With 
this definition, the active set is equivalent to 
the "violated set", and can easily be determined. 
Such a strategy is reasonable because the penalty 
trajectory algorithm is based on properties of the 
quadratic penalty function, and for a stif f ici'en^tly 
large value of p, the' set of constraints violated 
at x*(p) " is identical to the set of constraints * 
active at x* (Fiacco and McCormick, . 1968) . - After 
the first few iterations, the active set 'typically 
r^inains fixed for the rest of the computation. 

It was noted in the definition of the. algo- 
rithm that a special procedure is used when more 
than n constraints are violated at the beginning 
of an iteration. If more than ' n^ constraints are 
violated, and A has full rank, • the search direc- 
r .tion, ,p, is chosen to attempt' to' minimize 

I^c(xH-p) '^ , by computing p as the solution of 
* *T' 2 

minic + A pi . The search direction in this case 

is calculated as follows : . • , 

(1)- Fac^riie A in the form 

^ R 
rr * T* 

QQ 
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21 



-wher^ R is upper triangular and^on^-singular- 

^X2) Solved . Rp - -Y c, wheye tth columns of 
Y are^giveri.b^ the, first n : rows " of *Q/ ♦ \ " ' ' ' 

^"^^ computacion^l procedure is very similar, 
to the "calculation of the fSgrange iQulciplier ^♦ 
estimates,- and! relies on or.thogonal transformations ' 
to reduce to up.per, triangular form. 

/- ■ Normally, the condition that more than, n , 
'consttaints ar^. violated occurs because the current 
point isi>a' pooj: estimate of the solution, a^d* ^\ ^ 
does not **hold -at the next iterate. However, it is 
-conceivable^that this ^conditioa could exist even v , 
at x*» so that possibly every iteration might be 
* special. Jxn this case, the Hessian' matrix of the 



penalty • function is not ill-conditioned as the' 
penalty parameter approaches its limit. This 
special procedure has the same effect as choosing 
T P » « in the usual definition . of the algoritl^m, 
and hence Is equivalent to the Gauss-Newton method. 

3*2. ^ Selejtttion of the Active Set for the Barrier 



Tr a 1 e c t o r^Al go r i t hm 



The criteria for selecting the active set in 
the' barrier trajectory algorithm are not so 
.straightforward as in the penalty trajectory algo- 
rithm. Because the barrier trajectory algorithm 
is a feasible-point method, all problem constraints 
are satisfied at every iteration, and the subset- 
of active^^onstraints must be determined by analysis 
of the behavior of the. constraints as the computa- 
, tion proceeds. 

The procedure for determining the initial 
active set is described in Murray and Wright 
(1976b) , and' has been satisfactory on the ^examples 
tested. The active set tends to be altered only 
during the early iterations, because of the possibil- 
ity of misleading local indications that certain con- 
straints are active* The folloving< rules are used ■ 
to modify the active set at each iteration: 

(1) The cons^traint corresponding to the most 
negative Lagrange multiplier estimate (if one 
exists), is deleted, and the remaining multipliers 
are modified accordingly, 

(2) If -any constraint considered active ■ 
appears to be bounded away from zero as the solu- 
tion is approached, lit is^ deleted from the active 
set. This decision, gf' highly de pendent on scaling; 
further discussion is given in Murray and Wright 



■i 



(1976b). 

^ (3) If the. step-length argorithm-was re- 
stricted because, a supposedly in^ctive^ constfa^int 

■ was -v/iolated, .this constraint is .^ded toj/tlj^e . " 

•i ' *^ * ■ ' • ' -" ■ ■"'•t ■ . 
active set^'^at the' beginni'hg of thefnext iteration, 

, and will not be -deleted- during tTiatf Ration, - 

.regardless of <he sign of its m^^txplier estimate. 

(4) If the nx^er* of active, const rai'ftts 

exceeds,, n following the addition of (3), the con- 

'stiraint with-the, largest value of -c^Cx) *is 

deleted from the active set (a further scalifig- 

dependent decision). ♦ • * ' 

3.3. IJse of^ Orthogonal Factorizations ^ 

A factorization \^volving reduction of kjto 
> . ■ ■< 

triangular form by application of orthogonal / 

transformations is used in several steps of the 
trajectory algorithms. Such a ' factorization is 
" convenient and ^reliable for "computation, ' and has- 
many advantages over alternative procedures . For " 
example, in some a^^gorithms for solving PI, the 
matrix. A A -'is formed and used to solve linear 
systems; the poor numerical properties of this 
strategy are well-known (see Peters and Wilkinson, 
1970), especially Xhe p'ossible squaring pf the con- 
dition number that may result from the formation 
of A A- Furthermore, if the matrix A does not 
have full rankj. A A is singular, and some steps 
of the algorithm may then' i>e. undefined.^. 

Computation of the complete orthogonal fac- 
torization of' A means that steps of the trajectory 
algorithm can be defined (with relatively little 
extra work) even if A is rank— deficient (see 
Sections 3.3.1 and 3-3-2). Althouglr on>y^ the 
singular value decomposition can fully reveal the** 
closeness of the columns of A to linear dependence 
the complete orthogonal factorization j>rovides ade- 
quate jLnformation in many applications' (see Golub, 
Klema, and ^ewarr, 1976), since the conditioning, 
of the triangular matrix R serves to indicate - 
the ."conditioning** of AJ ■ " 



^.3.1. Calculation of a Lagrange multiplier 

estimate * ' " . ■ ' - 

The Lagrange multiplier estimate at each' 
iteration of ^^ the traj ectory algorithms is _^computed 
as a least-squares s^olution of. minllAX-gll^; this 
f irst-order^ estimate is acceptable, since the 
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local-^atc of cotJvergex>ce of the^'crajectory' methcRfs 



Vis not restricted %o th£!^rat€P of coryrergence of ' ^ 
the multipaiers (see Wright, 1976). The orthogonal 
£actori2atiotu^ -A can be used to . calculate a 
Bdnl^munirlengtfi ifeast-sqoareSj, solution, even-^when A 
^^^s^not have 'f'ull column rank; ^this alternative ^ 
is no^ pos^ble, with techniques^'tKat involve forraing-^- 



A A. 



When reducing A- to upper triangular form. 



colusnn interchanges are carried out so that the 
reduced* column of largest magnitude is selected 
-as the next colunm to be reduced; the matrix is 
consid^er^ -to be ^umeriCcSiy rank-def icienr if 
the norms of all unreduced cSlumns^are less than 
a prescribed tolerance. In this way, ail diagonal 
elements of R are "bounded helcw by the specified 
tolerance- Although ch-e ill-conditioning of R 
does not necessai^J^y reveal it^lfl^ the presence 
of a diagonal element that is very small relative 
-to the largest diagonal element, prevention of a 
too-small diagonal element is sufficient in many 
cases" to control serious ill-conditioning of R. 

c 

3.3.2. Calculation of the search direction 

"TJie search direction jin both tr^ajectory 
algorithms. is computed in two orthogonal comppnents 
— one in the range of the columns of ' A, the other 
in the corresponding null space. This definition 
results from the characterization that the search 
direction must satisfy a set ol" linear equality • 
constraints 9f the formr 



A p - b , 



(8) 



where b is some vector depending on the algorithm- 
If A 'has full rank, these equality constraints 
uniquely determine the component of p in the 
: range of the columns of - A, which is calculated 
as Tbllows. . J 

The orthogonal matrix Q that reduces .A 
to upper triangular form is ex;>licitly formed, 
by multiplying out the orthogonal tr^.sf ormations 
used in the reduction. Once Q is available* its 
rows, appropri-ately partitioned, provide the re- * 
'quired orthogonal bases for the range and null 
spaqe of the columns of A* ^ 

Ix^ the full rank case, it is straightforward, 
to compute the component of • p in the range of A. . 



Since, p « Yp^ + 2p^, the "equality constraints (8) 

a'^P - A^(Ypj^ + Zp^) - A^ Yp^ - R^p^ - h , - 

which gives p^^ as the solution of f non-singular 
triangular system. 

• If A is rank-def icientT t"He 'component p 

may be obtained as the least-squares solution of 
*T 2 

minOA p - bB , again using the complete oi'tl^gonal 
factorization of A. , . " 

In either case, the calculation x>f p is 
completely straightforward'- 



3^4. Approximation of the Hessian of the Lagrangian 
_ Function ; 

'Alternative techniques. 'for approximating the 

Hessian o£_ the Lagrang'ian function under various 

circumstances will riot be discussed in any deta^- 

(see Murray and ^Wright, 1976b, for such a discus- 

sion) , but we shall consider one key property of 

the Hessian approximation. 

The assumed second-order Kuhn-Tucker conditibns 
T 

imply that the matrix Z WZ must be positive 
definite at x*, where Z is defined in terms of 
A{x*) , and W Is the IJessian of/the Lagrangian 
function; however, W itself need not be posi'tive 
definite, or even non-singular, at x* *or in any' 
neighborhood of x*. ' 

Certain approaches to spying PI impose ' 
additional conditions on related matrices — for 
example, methods involving augmented Lagrangian 
functions ( see '^owell , 1969 ; Fletcher, 197A) 
require that the penalty parameter be large enough 
so that the Hessian of the augmented function is 
. positive definite., In both trajectory algorithms, 
however, only Che projected Hessian, Z SZ, must be 
po.sitive definite at every iteration, in order for 
the solutionis of the quadratic programs^QPl and 



(^2 "to be bounded. The vector 
of a linelcr system: 



T 

■■Z SZ p. 



N 



Z d 



is the solution 



(9>-"~ 



for some vector d, and^ should be the step to the, 
^minimum of a quadratic function related to the 
Lagrangian function. 



/ 
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Acccftding^y, the Mtflx used to calculate '.jp^^ 
is. alwaysT* maintained as nufaerically pc^itive defd- - 
nice. Wh^n Z SZ Is updated by a quasi-Nevton 
technlquie',, positive def initeness is maintained by ' 
^ updating the Cholesky factorization of t'he pro- 
. jected matrix, as^ in revised qua^i-Newton methods ^ 
- for unconstrained minimization (Gill and Murray, 
197Zb). When 2 S2 is obtained from* exacts or*,^ 
finite-difference, approximations to, second deriv- 
atives, the modified Cholesky factorization of 
T 

Z SZ is computed in. order to solve the system (9). 



In all c 



caAs,' 



A 



the matrix used to solve (9) fo^- p^ 
T 



is represented as LDL^,' where L is unit lower 
traingular, and - D is a diag^al matrix with all 
elevents sfrictly positive- Such a procedure 
assures that the.pbrtion of the search direction 
, in the null space of the columns of A. is always 
well-defined,^ and bounded. 

3-5. Srep^Length Algorithms 

3-5.1. Detection and c.orrection of unboundec^ 
decrease of penalty function 
Even when the problem PI has a bounded solu- 
tion, the corresponding penalty function or aug- 
mented Lagrangian function may be unbounded below, 
..for arbitrarily large values of the penalty .param- 
eter (Powell, ^ 1972) - Accordingly, when executing 
a one-dimensional minimization with respect Co a 
penalty function or aujgmented Lagrangian function^ 
care must be exercised to avoid the possibility 
talcf-ng an excessively large step. - 
In p^articular,^ the safeguarded cubic or / 
quadratic step-length algorithms (Gill and Murray, 
1974) used in the penalty trajectory algo;rithm 
require" specification of an upper bound on the 
step length- In the current implementation of 
the penalty trajectory- algorithm, thie upper 
bound is set to correspond to a step of "reason- 
able" size, rather than an extremely' large Value - 
In some cases, the upper bonnd may impos'e an 
unnecessary limit on the stepsize; however,- in 
igeneral such a restriction will cause no serious 

>loss of efficiency ^or the overall computation, ' 

. ✓ ■ * 

since t"he^ext Iteration usually corrects the 

possible poor^ scaling of the search direction. ^ 

This Gonservative strategy is considered to^^ 

justified by the extreme difficulties that result 

if an enormous step is taken because the penalty 
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• function is unbounded below; either the next - 
■ iterate tdmpleteiy unreasonable, or a J.arge 
■ nmSber of evaluation^ -.Of the problem functions are^ . 
required -before the unboundedness - is detected* 

In the penalty tifejectory algorithm, it is 
considei^ed that the. penalty function may be unbound- 
ed along the 'given direction if the step taken is 
the specif ied upper bound. ^ . Almost always, the. 
indicated unbound edness can be eliminated simp'ly 
by increasing the penalty parameter. " \ ^ 

• - * ' i * 

3.5.2. Special techniques for the barrier 

trajectory algorithm . ^ * ' c 

^ At each iteration of the barrier trajec*to4pr 

algorithm, a step-l|ngth algorithm is executed with 

rfespect to the logarithmic barrier function, which 
*thus serves ^s a ['merit function"- Several authors . 

(Fletcher and McCann, 1969" LaSdon, et al., 1973) 
have noted -;trhe deficiencies of standard step-length - 
algorithms, which are usually based on approximation 
by low-order polynomials, when applied to the* log- 
arithmic barrier function. Therefore, the step- 
length algorithm of the barrier trajectory method 
makes use of special techniques that exploit 'the 
known properties of the logarithmic barrier function 
to allow more efficient estimation of an appropriate 
step length; these techniques are based on simple 
approximating functions that contain a logarithmic 
singularity. Only a small amount of additional 
calculation is required to fit the special approx- 
imating functions, and their use leads to a signif- 
ijcant increase in efficiency of the bne-dimensional 
minimization, compare^ to standard procedures 
(Murray and Wright, 1976a). 

" ■ V ■ 

4. Conclusions . " ^ 

The pen^ty and barrier trajectory algorithms 
are based^ on the mathematical properties of the 
^ approach to x* of the successive iterates generated 
by the quadratic penalty function and logarithmic ■ 
barrier function, respectively. In theory, these 
algorithms have several desirable properties — for 
example, their derivation does not depend on condi- 
tions that hold only in^ a close neighborhood of x*, 
and their rate of convergence in the limit' is arbi- 
trarily close to that of .linearly constrained 
Lagrangian algorithms (described in Robinson, 1972; 
Rosen and Kreuser, 1972). * In practice, the current 



implementations of, the trajectory, algorithms have 
been successful- on many problems, deliberately 



ncluding examples for yhich the ideal assumptions 
%re violated. ^ The jresults thus far indicate that 
these algoij^tthtas confpare favorably w^th" similarly 
, careful implementations of other algorithms to' 
, solve PI (see Wright, 1976, for some typical 
tUJmerical results) .• . . ' \' 

.The overall arm of this^ paper has "been to' 
illustrate ^some of^the co\isiderations of numerical 
analysis chat enteV' the choice of computational 
procedures f^or selected aspects of the trajectory 
■ algorithms.' Numerical^ analysis may not play a 
significant rolfe ±n the process o'^ verifying that 
the"" expected behavior- of an algorithm under^ideal 
conditions' is 4ispla)j>ad nxanerically. How^^er, 
it is an elementary Tact of numerical analysis 
l?hat. theoretically equivalent mathematical pro- 
cedures do not yield- equivalent, or evpn Slose, 
numerical results,- and "it is. ati elementary fact 
'of life that hoped-for conditions .arre^not always 
■satisfied. Considerations o£ numerical 'analysis 
should, therefore, be applied to every aspect of 
the definition and irajJiementation of optimizatio'h 
algorithms in general. 
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Abstract ■ . * 

•' , • ^ ■ ■ 

The problem considered is the calcula- 
tion of the' least value of,, a generial dif- 
ferentiable function of several variables. 
A brief^ review of two types of- TOinimiza-. 
. tion Methods based on nonquadratic models 
is offered. The. first 'involves solving 
systems of linear * equations* in every it- 
eratipn. The second is derived . from a 
generalization of the standard conjugate 
gradient methods. ' / " . 

I, Introduction „ ;^ - 

■■ ■ .■ 

The problem to be discussed is- the 
computation of the xinconst;rained , minimum 
value of a general differentiable multi- 
variable function f(x). Most of the cur- 
rently available algorithms use *as a fun- 
.damental model the quadratic function 



(1) 



where Q is a positive definite matrix and 
it is the location of. the minimum solution. 
o,f (1). It is of interest to investigate 
•more general models' which may better re- 
flect the local behavior of general con- 
/ tiAuous functions- It' would be, for in- 
stance r.desirable to develop methods that 
.CM successfully handle cases where the 
.Hessian.- matrices are positive semi-defi- 
,nite-at the solution or elsewhere. 

An /interesting departure from model 
*(1) .can be accomplished if wgl a^sjome that 



f (X) =. P(q{x).) 



(2) 



where F is a. dif ferentiable, strictly in-, 
creasing., function of a single variable 
q>0. The gradient of f (x) is 



g(x) = F'Qi^S^^) 



where 



F' = 



dF 

dg; 



(3) 



(4) 



aind since F'>0 the minimum, of f (x) takes 
place at -Jf^x. Furthermore ^ f^) can "be 
minimized tin at most n steps rf we use: 



(a) a set of search directions mu- 
tually conjugate with regard to 
i.e. rdQ,d^,-- wd^_3_ satisfying 



'dt^Qd,j = Qri. j / and,/ 

. (b) the optimal steps 
the equations . * . 



1 ■ 



satisfying 



-(5) 



i+l 



= 0 



(6> 



for 



= 0,1, 2r3, - ,n-l 



This property of F.(q<x) ) becomes obvious 
if we ' make the observation that the non-^ 
linear function F'(<2(2S)^^ leaves the iso- 
contour cuirves of 'q(xT unchanged; alter- 
ing only the function values on these 
curves. ' ■ i 

Thus, the function f (x) = F Cq.C^)) can 
be^ minimized -in a 'finite number of steps' 
if we generate 0-cc3njugate -search direc- 
tions in the process of minimizing f (-x). 
Obviously, a standard conjugate-gradient 
algorithm- such as Fletcher-Reeves' or Da- 
vidon-PcSwell-Fletcher's will not be fi- 
nite (unless F(q) = q) since the. gradi- 
ent vectors of F(q(x)) are multiples of 
the gradients, of q{x) and this would al- 
ter the searcji directions generated by a 
standard conjugate gradient algorithm. 
However, .a* relatively simple modification 
of these algorithms will prov^^e this 
property. .. 

A pioneering work in this direction has- 
been done by Fried [1971] who has con- 
structed a. modified" F 1 etcher- Re eve s . meth^ 
od for minimi-zing f (x) of the form 



f tx) = 



y pal . ^ (7) 

Chere f is a constant valufe, • '■■ . 
Function (7) satisfies the relationship 



(8) 



and has . also been studied as cin optimiza- 
tion model by Jacobson-Qksraan [1972] and 
Kowalik-Raaakrishiian [1976] . They have 
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used equation (8) as a basis for an' opti- 
mization method which does not produce 
conjugate directions 'but is finite for 

t?). This method is brief ly .described in 
■ •Section 2. A similar f^class of functions 
has been also studieSs^by Cayison and Wong 

[1974] .-Section 3 shows a derivation of 
modified conjugate gradient methods mini- 
mizing f.{x) =. F(q(x))'and a ^detailed a- 
nalysis of -f C2c) = . Section 4 contains 
.numerical results and Section* 5 "is a con- 
clusion. .* * " 

* i< 

2. A method- based on the homogeneous* mo- ■ 
del > . - , 



We 'can rewrite (8) in the form 



f (X) = l(x-^)Tg(;c)+# 



•where y = 2 for function (1) 
•''if we define ■ . , 



f = Yf 



(•9) 



= (g(2c)'^.f (5£)'.-i) 



V = 5S g(x) 



. (10) 



(11) 



(12) 



(13) 



then equation (9) can be restated as:/ 



■ T ■ 

y a = V 



(14) 



I-f we compute X and v at n+2 distinct 
points (n is the size of the js vector) we 
get a system of linear ec^ations 



wh^re 



Y .= 





.Ya = w 




. (15) 












w «= 






'\ • 

T 

I • ^n+2 j 









arid df Y is non singular we" can solve (15X 
for a which contains the solution tq the 
minimization problem min f (x) . 

Clearly, it X3 possible to get an al- 
gorithm minimizing functions satisfyirig 
(8) in n+'2 steps. For general functions 
an iterative procedure can be constructed 
where a sequence of approximation^ 

~45* ^1' * * * '^i' ' ' ' generated such 3C.-^x 
waere x* is a local minimum'^. In this'^ro- 
cedure7 equation (15) becomes iterative ' 
Y^S- = V. ^d*^Y. is obtain^ ^y- removing 



one row' from Y.^,- and inserting the new 
row y*^ (2S.) . computed at 2^±. The vector w^ 
is also modified by one component. The 
method requires solving systems of . equa- 
tions which differ by one row. Solutions 
'to such systems can be obtained inexpen- 
sively and ' accurately by using different 
factorization methods (Kowalik-Ramakrish- 
nan [1976] and Kowalik-Kumar [1976] ). 

3^ A modified' conjugate-gradient method. 

In the method of conjugate gradients' 

of Fletcher and;*Reeves. [1974] the search 

direction is calc^alated. as^ a sum of two 

P 

vectors. 



d. 
^1 



-.gi+s.d.._^,i^i 



and 



^0 =--Ho 



(17) 



U8) 



The scalar multiplier can be deterr*- 

rained from the conjugacy condition 



T 



where 



- - ? • . - 
. ' ' l±'l^., 2i"2i-l, = 0 

Equations (13) . and (20T give 
and from " (17) * , ' " • 



(19) 



(20) 



(21) 



d. 
^1 



giXi-1- 

sLi^i^i 



Si-i 



(22) 



"Equation (22) is still suitable in the 
case when f (x) = F(q(x)) except^ that we 
have .to redefine It can b^ seen 



from '(22) that if wer'^efine 



Y. , - g-/F!-g. '/F'. , 
-'1-1 Hi' i ^1-1' 1-1 



123) 



then the search directions generated hf^. 
(22) will be collinear for any 'F(q(x))-- 
In' other words, if. C23) is used in equa- " 

tion (22), then ^n'-^l' ' * * * ' ' '5n-l- 
are scaled conjugate^'graaient '"directions" 
produced by the standard Fletcher-Reeves 
method for F(q(x)) = q(x) .- • 



Now 



Si^PiSi-Si-i' : 



sl:i^^i2i-2i-i> 



(24) 



where 



X 



>(25) 



Equation (24) can also be written as: 

(,26) 



I5ii 



•^i 



The modiified formulas (24) and (26) to 
compute Sj^ can be useful, if we know how* 
t;p . compute Pl- Clearly depends on the 

^^cRoice of F(q). Since available numerical 
results suggest -that' the fianction 
pq{x)^ may be a good model for uncoh-"^ 
strained optimization we will assume now 

. that - .. 



f'(x) = F{g{x)) = |q^/p>o 
The gradient of f (x^) is 

/ • ^i " ^iQ(Si-5)" 

and , ' " ' 



(27) 



(2^) 



i-t 



where 



Furthermore 



_ ^"1 
Pi " fT~ 



t ..i 

p 



(29) 



(30) 



^i-1. 



t-1 



(31) 



'and we assxame that t is an unknown value 
that has to be determined at each step of 
the iterative process. From (28) we have 



T ■ T A 

gi-ii?i-gi-i^ 

SiSi-.i-Si35 



(32) 



and from (31) and (33) 



1 ^ 



36) 



or 



f . 



\ ^ ^i^l^i^lgi^i .^^ (37), 



Introducing an abbreviated -notation for 
the coefficients of (37) we get 



a J = tb r+1 



(38) 



Equation. (38) (first time obtained by 
Fried [1971] ) is solved for a ' nontrivial 
iinique' root ti which in turrf is used to 
calculate pi from equation (31)*. . 

. As- indicated by Gpldfarb [1972] the 
Quasi-^-Newton methods can*-also be modified 
to minimize "f (x) = F(>q(x)) in a finite 
number of steps- The effect of nonline- 
ar ly scaling the objective fiinction on 
the Quasi-Newton - methods has been also 
• investigated by Spedicato [1976] - 

, computational results 

The extended. i:letcher-Reeves method 
(EFR) using f ormulas^(26) jand (31) , \where 
t is calculated by solving (38) /has been- 
tested on several standard f xihctions and 
compared with the classical Fletcher- 
Reeves method (FR) . An identical one- 
dimensional search routine has *been used 
in both methods. The programs - Have been 
' written in FORTRAN and. computations per- 
f03rmed in double- precision on IBM/360- 

The f oilowing probl^us . have- been tried- 
Test problem 1- (Quartic with singular 
Hessian) 

f(x) = (x^+10x2)^+5(x3-x^)^-t-(X2"-2x3)'^4^10(X3^-^4)^ 

■ * ■ ' 
The* starting point i? (3,-1,0,, 1) and the' 

fun:ction " has a minimum of zero ^ at 
(0,0,0,0). 

Test Problem. 2 (homogeneous guartic) 

f{x) = [%x^Qx+b^x+0-253^ 



and 



i-1 



^i 



Since 



But: 



g?-l^i-gI-A^i^^P^i-l 



TA^ T 



(33) 



(34) 



'4.5 


7 


3*5 


3^ 




'-0.5^ 


7 


14 


9 


a 




- 1 


3-5 


9 


8.5 


5 




-1.5 


3 


8 


5 


7j 




. 0 J 



Q = 



The starting point is (4,4/4,4) and the 



function- has a aiinimura of zero at 
^ (.5,-.5,,5>0) . 

Test ^problem 3 (Rosenbroc]c 's function.) 

f(s) = I0b(x^-X2)^+(1-X3^)2 ■ 

The starting point is (-1.2,1) and the 
function has- a minimum of zero at (1,1) . 

Test- problem 4 (a" Hilbert quadratic 
form) * 



f (x) = - hx^Hx ^, 



.k=2,3 



where 



ir j=l,2, ... ,5 



. The starting point is (-S.-s/. . . ' -3) . The 
^ solution is at (0,0, . . . ,0) - 

Test problem- 5 (4-dimensional Rosen- 
brpck's function) 

f(x) - 100Jx^.-X2)^+(l-x^)^+90(x^-x^)2 

+(1-X3)^+10.1[(X2-1)'^+(X^-1X2] , 

.+19.8{X2-1) (x^-1) 

The function has a minimum- of ' zero at 
(1,1,1,1) Table, shows the experimental 
results for test Problem I which is a 
quartic .function whose Hessian is. singu- 
lar (itas two zero Eigenvalues) at - the 
solution xT=(o,o/t},0) . It 'looks as in 
such cases f (x) = '^P is a better opl>imd.- 
zation model. This, may be due to the fact 
that f Cx); = ijP has a singular Hessian at 
the solution for p>l. • 
- ■ As - expected, t:he EFR method converges 
. after approximately n steps for the test 
problems="2 and 4 (Table III, rows. 3 to 
6) . It shotild Jb.e pointed out that . in or- ■. 
der to achieve convergence. £or these 
fui>ct ions, in approximately :n steps, the 
- EFR method had to be ^' implemented with an 
accurate one-dimensional search! More in- 
•teresting are tests involving general 
functions, '.-such as ROSENBROCK and 4-DI- 
MEJISIONAL ROSENBROCK.' In these -tests we 
have also implemented the EFR method with 
. an accurate one-dimensional search. Our 
objective has been to find if. EFR could 
generate better search directions; that 
is, ; converge in fewer iterations with 
the optimal step ^'ize strategy* 

Tables III andrxv, and row 3 of Table 
II. show sample runs. We have had, how- 
ever, a couple of cases where EFR ^\ .has 
been only marginally better or slightly 
outperformed "by the FR method. 

In most test cases the value of p.' 
.jdiffered significantly from 1 in the ini^ 
^- tial stage of optimization and approached 
1 in the final phase of convergence- For 
-example, the run shown in Table III p. 
varied from 138 to .572 between the se- 
cond and the twelfth^ iteration and was 
quite "Close to 1 afterwards. This was not 
the case with Problem- 1 where even at the 
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very end of the convergence process* "p. 
assumed values as large as 15. s i 

Conclusion ' . 

The preliminary results presented* in 
this paper suggest^ that the EFR method 
based on the mod^l -f (x)=lc {x)P,p>0 may 
^ deal better with funptions Whose Hessian 
is singular at some points aiorig the op-- 
timization .path or at the solution. 
^ There is no strong indication that MFR 
• based on this model can perform signifi- 
cantly better that FR when optimizing gen- 
eral nonlinear continuously .differenti- 
able. functions- j 

Another open problem is the influence^ 
of the one-dimensional search accuracy on 
^- the pertormance of the EFR method- 

^And the final comment. It would be in- 
teresting^ to design other -methods based 
^ on the moHel f{x)=F{q) where F'(g)>0 £or 
^g>0 and see of they 3ffer any advantage 
over f(x)=q- 





' TABLE' I ." 




Iteration ■ 
^ Number 


. FR 


EFR 


0 


-2150 X 10^ 


.2150 X 10-^ 


10 


.4793 X 10%, 


-1403 X 10"-^ 


20 


-1715 X 10"^ 


-9883 X 10"^ 


30 


-1882 X 10"^ 


.5172"x 10"^^ 


50 . \ 


.7786 X 10"^- 





Quartic with singular Hessian 
TABLE II 



Test 
Function 


FR 


^ EFR 


Solution 
Accuracjr 


Homogeneous 
■Quartic 


29*,?,03** 


5*,43^- 


* 10-^1 


Rosenbrock 


46*, 100** 


38*, 86** 




Hilbert 
k=2 


20* 


4* 




' Hilbert 
k^3 


25* ■ 


4* 




fM = QlL 


64* . 


7* 


10-7 


f (x) = 


52* 


6* -.69476 



^ *Number of iterations 

**Nxamber of function 
evaluations 



and gradient 



*q is a ^adratic function defined in 
problem 2 
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TABLE III 



4 



Iteration ^ 
Number 


FR 




0 


.19192*10^ 


.19192*10^ 


' 12 


.1482*10^ 


.7080*10"^.-. • 


^ 20 


.3491*10"^^ 


-4419^*10^'* 


25 


' .1009*10"-^ ^ 


V.1509*lJ^^^ 


30 


-7474*10"^ • 


.9292*10"-^'^ 


l-DIMENSIONAL ROSENBROCK . 

■ TABLE IV 


Iteration * 
Number 


FR ' 


" EFR 


> 

0 


.4166*10^ 


-4166*-10^ 


'20 


/842l*10-^ 


.7821*10^ 


60 


- .1624*10^ 


.57.87*1^)"*^ " 


80 


.2568*10'"-^ 


.33923*10"^ 



„ 4-DIMENSIONAL ROSENBROCK 
^ (-1.2,1,1-2,1) . 
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. . ' Abstract 

' Algorithms are given for the efficient solu- 
. tion of the class of nonlinear integer programs with 
separable. convex objectives and totally unimodular 
constraints. Because of the special structure' of 
this proiDleni class, the integrality constraints on 
the variables can be easily handled. In fact^ the 
integrality constraints actually^make the problem 
"easier"'than its continuous version, for ia^the 
case that bounds are available on the problem vari- 
ables, the first of the proposed algorithms yields 
the optimal solution by the solution of a single , 
.easily-generated linear prograni. For the cases in * 
which bounds are not*avaiIable for the variables or 
the sum of the variable^ranges is very large, other 
algorithms are discussed that yield the solution 
after a finite number of linear programs and require 
less storage .than the first algorithm, 

1. Introduction * 



Consider the nonlinear integer program*''" 



U.l) 



' min 

X 



S'.t. 



n 



f,(x.) 



Ax = b, X > 0; X integer 



where x = (x 



•1^ 



c 



(each f^ is convex on a ' 
closed interval C [0,+oo) and A is totally 

'''When modified in the obvious feshion; the algo- 
rithm to be described below can be used to soIvq. 
more general problems in which (1) the constraints 
Ax = b are replaced by a system of equations and 
inequalities with a totally unimodular constraint 
matrix, and (2) integer upper and lower bounds on 
individual variables are added. Alternatively, 
isuch constraints can be converted Into the ionh 
(1.1) thrcugh-the addition of, integer slack variables, 
and the resulting coefficient matrix will be -totally 
unimodular. ■ " ■ ' 
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unimodular mXn matrix. The Intervals B^, which 
are assumed to be given, are assumed to cover the 

_ feasible set of (1.1) in the sense that i2 ^ {x|Ax'= 
b,x>0.^x integer} £ {x| x£€^%: i=l, . . . ,n}. (N9 dif- 

^ ferentiability or continuity? properties are needed or 
a^ssumed for the 'f^, but convexity of nf^ on im- 
plies' con^nuity on t^ie interior of ,Bj^. ) 

Problems of the class (1. 1).- arise in logistic 
and personnel assignment applications, and have 
. /been the subject of a riumber of studies [1], [4], [6], 
. \J\t [S] ^^3^ <^eai with the special case in which 
■ - ■* . ' n 

Ax = b consists of the single constraint ^x^. = K. 

(see also [5«] for. an algorithm for this spe^Tlal casej. 
Dantzig [3, p. 498] considers the case in which the 
constraints Ax = b correspond to supply-demand 
constraints in a bi-partite network. 

The problem class (1.1) has the remarkable 
property (shown in [5]) tlTdt Lheln teg rality constraints 
actually make the problem easier to solve than its 
corresponding continuous version^ 

Specifically, it was shown in [5], that if 

.J there exist known non -negative "integers i!^,u^ such 
that Bi = [ii,u^] for i = 1, . . . ^n (i. e. , tihere e5ci:st 
knowr* bounds for the feasible set of (1.1)), then 
(1.1) may be solved by solving the single linear pro- 
gram constructed by (1) replacing each f^ by an ap- 

•c propriate piecewise linear "approximaUon" and (2) ' 
deleting the integrality constraints. It was also, 
shown in [5] that, if the intervals are infinite 
(or if the sum of the ranges of the x^ is very large), 
an algorithm based on the solution of«a finite number 
of similarly constructed linear programs is guaran- 
teed to yield the optimal solution within a- finite num- 
ber of iterations, under the'assumptipa that the.giVen 
■fH-oblejn of class (1.1) has an opfimal solution.' In ■■ 

■ "this report we conside]: in" more .depth the question 
of the. trade-offs between number of LP'S:solved, 
storage required, and number of function evaluations 

: for four specific algorithms of*;the general types de-' 
scribed previously. A numerical example is discus- • 
sed in Section 4. ■ . . 
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2. . Basic Algorithms * 

In order to describe in a-^ompact manner 
algorithms for the problem (1. l^^^w^ wilKlntroduce 
some notation to represent certain piecewise-linear 
approximations to (J.l). .Letting (i=l, . . . ,n) be 
finite sets of non-negative iotege;"s, we define a 
llneay programming approximatiqa to (1.1) as the 
problem . P( R^, . . . , R^) =s.given by : ' 

2 Z 

i=l 



min 



U R, 



(2.1) 



s. t. Ax = b/ X > 0 



2 \^^j = l(i=l,..,,n) 



> 0 



Je R^ 



^he problem P(Rp . .-s^Rj^) can be thought of.^s a 
^separable programming approximation to (1.1) in: 
. which the integrality constraints are deleted and 
the brea kpoint- sets \ are given by the sets R 
(i=I, . . . ,n). The significance of this LP approxi- 

• mationy as- shown in [5], i^. that (1) an extreme point 

of (2.1) will have integer-valued for i =1 n. 

and (2) if ,xj*'(i=l,. . . ,n) is part of an optimal solu- 
tion (x*,X*) of (2.1^ and the breakpoint-sets" 
have the property that (for 1=1, . . . , n) 

:(2. 2)' [{x*-l.x^,x*+l} n C R^ , 

then an optimal, solution to {l.l)^is obtained by set- 
ting x^ =: x^ (i=l, : . . ,n). (It should be noted that 
it is essential, to employ a "separable programming 
approximation" to (1.1);. since other'piecewise- 
llnear approximations' that agree with f^ at the . 
. integer points in B^(i=l, . . . ,n) may not have the 
required extreme point integrality property - see 



. . We will how consider three algorithms for 
; (I.I) corresponding to' three, different procedures for"" 
constructing the R^- 

Algorithm 1 (Single LP, maximal R^) 

This algorithm is applicable only if there 
are known bounds ^^,u^ (which will, without loss , 
of generality, be assumed to be non-negative inte- 
gers) sucfithat = i.e. if x- is feasible 
for (1.1). then < < u^ for i = 1, . . :.>n. It 
has the sdvanfage that it yields a solution to,(l. 1)" 
via the solution of a single easily-constructed ■ 
linear program. -Specifically^ let each con- * 
- sis.t.of the-integers in the'interval ^[^^jU^], and - 
solve, the LP (2.1), then note that the..optimality 
\ . condition (2. 2) is satisfied by x* if (x*,y*) is^ 
; an optimal extreme point of the LP (if (2.1) is in-' 
feasible^ then so is (i.l». ' 

Algorithm 1 provides a very efficient, one- ■ 
•step approach to the solution of (1. 1) as long as the 
/ problem P([£i,uJ,. . . ,[l^,u^]),\v;hich wUl have' - 
n" ' ^ 
2n +^(u^-^'^) variables and m + 2n corvstraint^. 



does not exceed the capaci^y of the available linear 
programming algorithm. In this- regard, it should be* 
noted that many commerical LP "packages", have ' 
separable programming and/or generalized upper 
■ bound capabilities that take advantage of the special 
structure of (2. 1). It is also possible to modify the 
data handling in the simplex algorithm so that a dis- 
tinct column is not needed for each \^ since, for 
a given i , the columns corresponding' to the' X.^ j 
can differ only in jtwo entries. Furthermore, if tlie 
constraints Ax = b can be given a network.interpre- 
,. tation, efficient algorithms for network optimization 
can be applied ra^ther than the ordinary simplex algo- 
rithm (see, for example, [2:]), 

However, if known bounds i^,u^ are not 
available or if the attempted implementation of Algo- 
rithm 1 would lead to storage proble^ms, then alterna- 
tive approaches are possible because the optimality . 
condiUons (2. Z) do not require a "full grid" of points: 
The next two algorithms take advantage of this. fact. 
To get starting "grids" for the next two algorithms, . 
two cases should be considered: (1) if bounds ^^^u. 
are known, then the initial breakpoint- sets R^^) 
needed for the algorithms^ can be ta^en as any inte- ' 
^ gqr sets containing at least and u^, and (-2) if 
fLhite bounds are not available for- all variablesV let 
X be an extreme point of {x|Ax=b,x> 0} (suc'h an 
X may be determined by the simplex method, and it 
will be integer), and set r[^^ to {^i,Xi,Ui}, where 
and u^ (i=l, . . . , n) are estimates (vvhich need not 
be rigorous) for lower and'upper bounds on an opti--. . 
mal solution of (i.l). (If a good guess is available 
for the optimal solution of (1.1), the corresponding 
Ijreakpoints should also be included in the' rCO)^ ) 
In both cases, the initial LP considered wili be 
feasible if and only if (1.1) is. feasible, so we 
will assume in the algorithms below , that feasi- 
bility has been established. ^ <, - 

Algorithm 2 (Multiple- LP's, intermediate R^'s) ' 
At the kth iteraUon (k=oa,2, . . . ), we as-: , 
sume that 'a feasible solution x^^^ to (1. 1) has been. . ' 



obtained by solving the problem 



If the optimality conditions (2. 2) are satisfied by 
X* = x',''^ and,-R^ = r[''^,' then teminate with x* ^ 
as optimal solution of (1. 1). . Otherwise obtain new 
breakpoint- sets R: by adding to each R^.'^) the 
,.(k) 



points of [{xj,';-'.l,x[''';xj''^+l]} n B^]\r|^\ (these 



are the "points "missing" from the optimality condi-, 
tions) and let ;^e (k.+l)st iterate x^'^''"^^ be the value 
of X in an optimal extreme point bf 
will be integer). 



We also assume fhat the algorithm tests x^ 



'for optimality in P(r|^"^^^ 

by declaring x^ - ^ optimal for (1.1), if this test is 
satisfied. 



,R^ ) and terminates 



\ 
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(Since (x^^'^X^*^*) would nQhnally be used 



as a startin^g basic feasible solutftn for 

. ,R^^t^'), this is a -natural assumption. ) ■ 

Algorithm 3 (Multiple LP^s, mlnimar R^*s) 

• Proceed as in Algorithm 2, except that if 
x^^^ and the R^^^ do not satisfy the opUmality 
conditions of (2. Z), then the breakpoint-sets r'^ 
for the next iteration are taken to b§^the sets 

{x|^^X^x[^^x[^>+l} n B^.(i=l,v. -;,^ • ■ 

Algorithms 2 and 3, whi.ch represent oppo- 
site ends of the column-generation spectrum, will 
converge to an optimal solution of (1.1) in a fipite 
number of iterations, provided that (LI) actually • 
ha s^ an optimal solution; This result follows from 
a finiteness theorem [5] for integer programs with 
separable convex objective functions. These algo- 
rithms also in general avoid the problem generation 
and storage problems that may occur in some cases 
in Algorithm 1. In certain instances, however, dif- 
ficulties with storage limitations and/or speed of 
convergence might also arise with Algorithms 2 and 
3, and some additional" computational refinements 
are described in the following section. 

3,' Some Computationally Useful Modifications 

Although the algorithms of the previous sec- 
tion are guaranteed to display either- one-step or 
finite convergence under rather weak hypotheses, 
theoretical finite convergence of course does not 
necessarily imply that an optimal solution will be' 
obtained within the time or storage av^^labie to 
solve the problem. 

If Algorithm 1 can be employed' without ex- 
ceeding- the capacity of the available LP or net- 
work code, then/,time and storage ^will not be prob- 
lems unless y (u.-£ J turns out very large^Q 

the other hano^ Algorithm 2, starts out.with rela 
tlyel]/ small breakpoint* sets, but*there is ap'con- 
'^rm^n the size of these sets, and thus no guar- . ' 
antee that problem size limits might not be ex- . ^\ . 
ceeded. ■ 

- Although Algorithm 3 em|)loys the minimal ' 
breakpoint- sets needed' to establish optimality, 
one would not expect it to be very efficient^ since 
the value of a variable can change by at most one- 
unit at each iteration, and since much of the com-: ' 
puted information on values of the f^ may be djs- ' 
carded at each iteration. . In addition, slow con- . 
vergence might also occur in Algorithm 2 in the 'case 
that lower 'and 'Upper bounds are not known -and .at 
least one p£ the estimate's /j^^Uj. is consistently • 
violated by the iterates, since Algorithm 2 allows 
only a single unit change beyond the estimated 
bounds at "each iteration. - „ * " 



Algoritjim 4, to be desrrlbed below (see 
also Figure 1) avoids these . potential problems, 
and also makes use of the possibility of 
a lower bound on the optimal solution of (1.1). To 
set up the linear program for determining a lower 
bound, let x'*^)^ denote the. optimal solution of the 
most^ recent LP solved, ^and let 'be a set such' 

that,^ for each i , either x[^^ c Rj"^ or x$^'-l € .R! , 
A i 1 • i 1 ' 

and such that j t^RJ implies (j+l) < (while any 

set of breakpoints with these properties' will be suit 

able for lower bound generation, larger R| will 

yield, larger LP's and^ in general, better lower 

.bounds). ^ ^ 



A lower bound on the optimal value of (1. 1)' 
may then be obtained by- solving the -following LP, " 
since the objective function of the LP is no greater 
than the objective function of (1.1) on. the. feasible 
set 



min 
x,y,6 



i=l 



(3.1) 



s,t, Ax = b,x>0- ... 

> + S^^.(f^(j+l)^f^(j)). 



X. = j + 6. . 

^i 



1,-,. .^n) 



(Of course, the' lower bound generated by solving • 
(3.1) may turn out to be in which case it would 
not be useful. In many cases, however, additional 
infprmation such as nqn -negativity can be included 
in the lower-bounding LP in order to prevent un- 
boundedness. For example, if the functions fi(x^) 
are known to be non -negative for non-negative 
(in many applications the f^ are exponentials or 
posynomials), then the additional constraints y^ > 0, 
when added to the constraints of (3.1) will prevent, 
unboundedness of the objective function. ) . Since 
Algorithirts::2:.aad ;3. -(and Al-go.rithm ■4': belo';^). ►generate 
feasible solutions to ll A)y it is possible to us^ a 
lower bound on the optimal value of (l.i) in a'termW 
nation criterion.- if one sets an optimality tolerance 
on^he gap between the Lower bound jand the. value of 
the best. feasible'^solutiph-^ (In ger^ai, the feasible 
sol"ution<^th the best objective v^ue .will beo the ' 
last iterate^ .but an "exception to^his might. occ§3?;^ii 
an optimal solution (x,2^6j to (3.1) had the 6>o]^ 
erty that x was integer.: In general, a solu'tioTt^' 
(3. 1) will not have this /integrality property, but:if H 
doeSy.'then -X - will be a fee^ible solution of (1.1), 
and m^y have an objective function value better than 
that of the last iterate, in which case this^iterate 
should^be replaced by' x , Note that if x is, integer 

and . 2 f.(x) coincides with the optimal valxie' of " 

(3,1), then x . solves (Ul). See Figure 1 for the de- 
tails. of how these possibilities may be taken into 
account.) . . 
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Figure i. Flowchart fior Algorithm 4 
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The statement of Algorithm 4 assumes that 
the following parameters have been supplied: (a) a 
bound on the total number of j Variables that 
will be allowed in any L?\ (b) incre- 
ments ^if^i used for revising lower and up- 
per bound estimates (these are not used If rigorous^ 
Uower and upper bounds are .known), (c) an optimal- 
ity tolerance e and a parameter establishing the 
frequency of the lower bound com^putatlons (this 
could be based on clock time or number of itera- 
tions). 

Algorithm 4 (Multiple LP's, bounded stor- 
age, optimality tolerance) 

Proceed as in Algorithm 3, except that: 

(a) when the update of the would violate 
the bound on the number of remove from the 
breakpoint sets Tprlor to the update, 
all lndlces other than those corresponding 
to the current values of the upper and lower 
bound estimates: 

(b) when an iterate violates the current value 
^1 of the lower bound estimate, l| is up- 
dated to max{p^i J-or^}, and when an iter-- 

- ^te violates the current value u[ of the 
upper bound estimate, u^ is updated to 

4 + Pi ^ ' 

(c) . _ ^ a lower bound is periodically computed by 

solving a problem of the ^^rm <3.1), and the 
algorithm is, terminated if th^ objective func- 
tion value of the best fea«?ible solution ob- 
tained thus far lies within c of the lower 
bound (if. the t^^ problegn of the form (3. 1) 

' has an optimal value z > -<», the con- 
n • ^ ■ — 

stralnt Jj Y. > 2^ ma^ be added to the 

-1=1 ^ ' . : 

(t+l)st problem^ of the form (3.1) in order to 
guarantee monotonicity of the lower bounds ; 
• note that the: dual simplex algorithm may be 
. ' • used with the o'ptimal .solution, from .the / t^^ : ■ 
/jjioblem serving as. the initial^plutlon.pf *; ; 
-■ the dual of the . (t+l)st problism).. . .ir 



For notational convenience in. the flowchart 
• . ■ ; ■ n . - , , ■ 

for Algorithm 4, ^^C^J is denoted by ^(x) and 
, - 1=1 ' ■ 

r P(rJ^^.:.,r[^^V Is denoted by P^^'. 

4. : Numerl_cal Example .. 

- . In this section we present a numericaT ex- 
ample to illustrate the algorithmic ideas introduced 
in the previous sections. 

The problem dealt with has the form ' 

v.min ]j Vr (Uq ) : ■ ^ \ . " 

' ■M=i: ■ ■' 

10 - . 15 . 
^4.1) s.t; 2 = b,, ^ X. = b ^ 

i=l . Vi=^ 
0 <x<u, X integer 



where the data and optimal solution, x are given 
in Table 1. Using -the ^'complete grid" approach of ' 
Algorithm 1, the resulting\p has 17 equations and 
262 variables,. and the use of a small, locally- ^ 
written LP package yielded the optimal solution x 
of Table 1. . . 

* By contrast, a column -generation procedure 
of the type described in Algorithm 2 started with ^0 
variables^and terminated with an optimal solution ' 
(X* of Table 1) in 7 interations,. the final LP solvejJ 
having 131 variables. . 
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- = 75, = 


67, optimal value = 7,7386 



\ . . Table 1. Data for Numerical Example 
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EXTREME POINT RANKING ALGORITHMS: 
A COMPUTATIONAL SURVEY. 



• Patrick G. McKeown' 
The University of Georgia 



.ABSTRACT ^ 

Since it ^as been long known that the optimal so- 
lution to the minimization of a concave objective 
fuxfetion over a convex set will occxir at an ex- r 
treme point of the convex set, one method of solv- 
ing, this type of problem is to rank these extreme 
points. In the case where the objective function 
in nonlinear 2tfld the constraint set is linear, 
there have been nuinexoiis articles, more conceptual 
than computational, on the application of an ex- 
treme point ranking algorithm as. a solution proce- 
dure. In this paper, we will review the useful- 
ness of this* general type of procedure to various 
problems by attempting to combine the available 
cooiputati'onaJL literature with computational re- 
sults of the" author that have not been previously 
presented. 

I i Introduction , 

This paper will be concerned with problems of the 
following ^genered. type: 

min f (x) *. ^ 

* , s,t, .xcs ■ ■ • - . (P) ' 

' ■ where. S « .fx|Ax»b,o^O}', . ,. 

'■ and .f Cx)'-is. concave. .A is assumed to be • ' 
m X n^ b is m x 1, and x is n x'ltv 

; . ^ ^. 

It was ihown by Hirsch and- Hoffman ' [12] ■that'^an 
optimal solution to P would occur at an extrem^ 
point or vertex of S. Hence, ^ to find ciri optimal, 
solution to P it is "only** necess'ary to search the 
vertices of P -until an optimal solution is found 
and proved, 'if f (x) is linear, i.e.^ 

■ n . 

■ ■ ' . . f (X) = Z C X , . 

then the well-knpwn simplex method is a very effi- 
cient procedure for carrying out this seaurch. Ho\* 
ever, if f (x); is nonlinear, say, quadratic or in- 
teger, there does not exist a simplex type of al- 
.gorithm. It is the nonlinear case that we shall 
be concerned with here.' ^ 

Since no "direct" optimization techniqpaes exist 
for the case where *fCx)^ds nonlinear, we' shall 
lock at two approaches to searching the extreme 
points of S. First, one might wish to use a lin- 
ear xuader approximation of f (x) , say L (x) , such 
that I.(x>£f.(x) xgS. . Tn this case, to show that x*- 
is an optimal --solution to,P, we need only rank the' 
.vertices- of S until the vertex x** is found such" 



■ that LCx**J>^f (x*) . At this point > all vertices 
that could possibly be optimal have been ranked. 
This is proved by Cabot and Freincix [3] ♦ 

The second case applies to the case wj^ere f (x) is 
a sum of a linear and a nonlinear portion, i.e., 
n * • 

f (x) « Z C;X. , + g(x) 

j«l ^ ^ ' . f 

where gCx) ^0 for x >^ 0 
In this situation , one 'may seek to find a lower 
bound on gCx) , say G. Then we may use the linear 
portion of f (x) plus the lower bound as an lander 
approximation of f (x) . Obviqus^ly , for x >^ 0 , 
f(x) >^ C X 4- G. Hence, if x* is optimal for P, 
then we need only rank the vertices of S untilv a 
point x* is found sucl? that C^x? + G f^x*). This' 
proves that x* is optimal. The best example of 
this is the fixed charge problem. 1 

In ord,er to rank the extreme points of is, we need 
to use in both cases above a result also first, 
proved by Murty [20] as Theorem 1 below: 

Theorem 1: ' If Ej^' ^""•'^k the first .K ver- 
tices o^ a linear uJlaer approximation problem. ^ 
which are ranked in: nondecreasing order* according 
.to their objective vaQue, then vertex must he 

adjacent to one of E « • 



^2" 



V 



simply put, this says that vertex 2 will be adja- 
cent to the "optimal solution to the linear under 
approxijnatj-on- and vertex 3 will be adjacent to ver- 
tex 1 or vertex 2. This, then, gives us a proce- r 
dure for ranking the vertices if atll adjacent ver- 
tices cctn be found. It is this '^if" that quite 
possibly has accounted for the few number of com- 
putational papers relative to the numl^er of con- • 
cept ual works. This comes about, due to the possi-^ 
biHty o'f degeneracy in S. If .S is degenerate, 
then there may* exist iliultiple bases for the same 
vertex.. This implies that all sucl^ bases must be 
available before one can be sure that all adjacent 
vertices have been found. Finding all such bases' 
for finding 2Uid "scanning" all adjacent vertices 
can be quite cumbersome. As we shall see later, a 
recent application of Chemikova'':^ work {*5J has ^ 
been shown to be a way ardund the problem of de- 
generacy- This will be discussed in more detail 
when the fixed charge problem is explored. 

The literature has been found to refer ^to basically 
four tyj^s of problems: 

(i) fixed charge problems. 
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Cii) traveling salesman, 

(iii) quadratic assignment, and 

Civ) concave quadratic programming problems* 

We shall briefly discuss the conceptual background 
of each problem and then present and discuss the 
available computational applications of the ex- 
treme point ranking procedure to each problem. 
Where ' appropriate r we will present previously un- 
published womc of .the author on th^ ^>roblejn at 
hand. Finally, we will attempt to draw coaciu-'* 
sions about the efficiency of this typp of procc- - 
dure. 

II. The Fixed Cheurge Problem 

One of the first types of problems recognized as 
being of the form specified as P previously is the 
lineeir fixed charge problem. This problem is for- 
mulated as P_ below: 
F 



Min 



c x •♦- f y 



S.t. XCS 



if X. 
if x^ = 



In this case, f and y are nxl eind all other dimen- 
sions are as before. The f values are' the f.ixed 
or set-up" costs while the c values are the. con- 
tinuous costs, Hirsch and Dant;zig [10, 11] first 
formulated and recognised that an optimal so- 
lution of Pp would occur at an extreme point of s 
. if the f aije non-negative. 

A special subset of the f^ixed charge problem is 
the fijfed charge transportation problem CFCTP) 
where the A matrix JtcJces on the form of the Hitch- 
cock Transportationr Problem constraint set, 'i.e* ^ 

■ . : . " ' . ■ "/ V- ■' ' ' ■ ' .' 

■ " " ^" ^- * ^ ~ 1,.-,. - ^N 

i=l 
N 



. and 



Z X. . = s . 

M N 
Z s. « Z d. 



i = 1,*-*,M 



.i=l 



Here, M = ntimber of supply points and N = nimiber 
of demand points^ Balinski [2] showed that a 
linear under approximation, of . the objective 
function of the FCTP" could be found by first 
letting u. . = Min .{s., d.} eind then setting ■, ^ 
^ M N ^ 
. L_(x) 2:. Z CC, + f^^/u_) X. 
i=l j=l 
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-Although Hirsch and Dantsig had formulated the 
fixed charge problem and Balxnski had found ap- 
proximate solutions earlier, Wurty [20] first- 
suggested the use of .extreme point ranking algo-. 
rithm for solving this. class of problems. He' 
showed that if a lower botmd on the -sum of fixed 
charges could be found, say F , then the optimal 
solution to Pp, (x*,y*), could be found; by ranking 
the vertices of the corresponding continuous proh- ■ 
iem (all. f . = 0) until 'a^ooint x is found such 
that C X C X* + f y.. The values of ■(x*,y*) 

may be found by checking each extreme po^t to de- 
termiire whether a lower value of C x + f y existed. - 



•When the point x is found, the solution (x*,y*) 
is optimal. Murty did not discuss amy. computa- 
tional results and left .several unresolved prob- 
lems with the solution pcocedure. * 

The first unresolved problem was to find F^. Murty 
suggested that the m smallest fixed charges be 
summed. This may be easily seen to be inadequate 
for problems with greater-than constraints or for 
degenerate problems. This method also does not 
attempt to find a lower bound that is feasible. 

Secondly, Murty did not discuss cuT^dequate method 
of h2indling degeneracy in finding adjacent extreme 
points other than determining all bases and apply- 
ing the simplex method to each\)ne in turn. 

Although Murty did not present computational re- 
sults of using his extreme point ranking method, he 
did hypothesize that it 'would probably be more ef- 
ficient^in those cases where the continuous por- 
tion (C X) dominated the fixed portion (f y) . This 
was later shown by McKeown in his dissertation [16] 
cind in a later article [17J. He. effected an imple- 
mentation of Murty 's procedure by resolving £he 
two problems mentioned earlier. He showed that a 
lower Bound on the fixed charges, F^, could be 
fotind by solving by linear programming the set- 
covering problem P below. 

- ..^ " • 

rixn F. = Z f.y. • 

n • ■ . 



S.t, 



Z 6. .y. 
=1 3 



j=l 



3 



y. > 0, 



where 5 . 



3-3 



4i 



and 



a. . > 0 
a^^ < 0 
■ ^-^ ~ 

1 (except for the. FCTP wheref/S'. 
* ■ 3 - ' <3[ "1 

values greater than 1 were iisea • 

to account for the number ;^of -^qI Is 

. ^ in' a row necess£try to possibly 

^ achieve feasibiJKbty) . 

This method was fo;ind to yield better (larger) val- 

. ues of f^ than that originally suggested by Murty 

while also handling the problems with greater-than 

constraints and degeneracy mentioned eaurlier. ' . 

. To handle the problem caused by degeneracy of S in 
■ >finding adjacent vertices, a modification of work 
by Chemikova [5 J was used to determine adjacent 
vertices of a degenerate vertex [25] . * ' . 

Computational experience with this implementation 
of - Murty,* s suggestion bore out the original ^ con- 
jecture that ease of ^ solution would largely pe a 
function of the relative size of the fixed atd con- 
tinuous portions of th^ objective function. Vwo . 
types of problems were tested.' First, a group of 
general linear fixed chsurge problems. generated by 
Steinberg [26] were run using a FORTRAN cofie on an 
IBM 360/17S. These were 5x10 problemis with equal-, 
to constraiints which-were randomly generated^ such 
that 0 £ C. < 20 and 0 5 d. <^.999. Five of these 
problems wSre tested and solved . regardless of re- 
lative costs . " • . 

In^the second case,. a set of nine fixed charge 
trcirisportation problems originally generated by 
Gray J[9] were tested. Xn this Ccise, these prpblems ■ 
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ranged from M-3 and N-4 to M-6 and N«8. Three" of 
these problems had a relatively large continuous 
portion while the remaining six were fixed charge 
dominant. In the former case, the procedure was 
quite efficient regardless of problem size while 
in the letter case, the algorithm proved to be in- 
efficient in that none of, the six could be solved j 
due to storage overruns. • 

In the later article [17] , McKeown expanded the * 
computational' experience by using more variations 
of Steinberg's problems, i.e., 10x20 's and prob- 
lem^ with greater* than constraints. In these, 
problems, results were encouraging as long as the 
relative cost sizes were favorable. For the prob- 
lems with equal-to constraints, .the fixed costs 
dominated 'due to M structural variables . always 
being, basic. However, in the problems with ' 
greater- than constraints > the reverse Was true and 
the results show^ that extreme point ranking was 
much more efficient for this latter cl^ss of prob- 
lems. ^.He also tested for the effect of degeneracy 
on FCTP's by comparing degenerate versions of 
originally non-degenerate problems. The results 
appeared to show that degeneracy is not a problem - 
as* long as the relative costs are favorable. 

In summary, Marty's extreme point rauiking proce- 
dure does appear to be, efficient in those cases 
where the fixed cost portion is small compared to 
the continuous' portion - 

Two articles have appeared that use cutting plane 
variations of Murty's procedure- Cabot [4] sug- 
•gested the use of Tui [29]. cuts. The original Tui 
algorithm involved determining a local minimxrri, 
say X, over s. Then a hyperpleine is passed 
through the convex poly tope in sxich a way that all 
extreme points of s with value greater than the 
local minimum, x, are excluded. These are com- 
bined with the linear under approximation for fix- 
ed charge transportation problems , 

He used two problems .generated • by Gray .for his 
testing. Both problems had M=4 and N=6. Problem 
1 was continuous cost dominant while. Problem 2 was., 
fixed cpst dominant. Before using the Tui cuts, 
Cabot attempted to 'solve both problems via extreme 
point ranking using L^(x) . As wouId.be expected, 
he quite easily solved Problem 1 but was. unable to 
prove optimality for Problem 2 after ranking over 
400 extreme^ points. He then used a combined Tui 
Cut - extreme point ranking procedure to solve 
both problems with equal efficiency. This insen- 
sitivity to relative costs became more apparent 
when he devised 30 test problems by randomly gene- 
rating new objective functions for Problems 1 and ^ 
2 »where the ratio of fixed costs ^to continuous 
cost ranged from .5 to 20p. For these problems 
his procedure appeared to be lequally efficienot 
regardless of the ratio of fixed to, continuous 
costs in. that he solved 23 of the 30 test problems* 
He handled degeneracy by using a pertubation 
-scheme^ which while resolving degeneracy, intro- • 
duced numerous additional extreme points. 

In another paper, Taha [28] combined extreme point 
ranking with "Glover [8] cuts" to solve general , 
lineaur fixed charge problems. He defines a Glover. 



Cut to be one which separates a given extreme point 
from the convex polytope. He used as a linear- 
under approximation the continuous portion of ob- 
jective function, i.e., 

n • . , . 

' L cx) - I c,x ; , 

and used these Glover Cuts to reduce the number of 
extreme poix^ts to be ranked. In addition, he used 
a technique suggested by Balas HI of dropping con- 
straints associated with the degenerate basic vari- 
able to redefine the poly tope. It appears that ^ 
this was done to insure that the Glover Cut was* de- 
fined rather than to find adjacent extreme points. 
This procedure aiso tended .to generate additional 
extreme points to be considered. Taha solved ran- 
domly generated problems of size as 'large as 15x20 
with 0 < C. £ 800 and O-^ F. < 100 in an average of 
47 seconds^on an IBM 360/50? As may be noted, 
these problems appear to be continuous cost domi- 
nated. This conjecture seems to be borne out by 
the second set of test problems where for 0 <^ F . <_ " 
300, we find that the solution times have gone dp 
by a factor .of four. Since these problems were 
randomly generated, it is highly unlikely that any 
were degenerate so we do not Jcnow the possible r_ 
effect of degeneracy on the solution procedure. 

One point that Taha makes 'is that' the use of a lin- 
ear under approximation for general problems simi- 
. lar to that suggested bySalinski [2], i.e^. , \ 
n • ^ 

L .(x). «• Z^iC. r./u.) X, ^ 
. . g V 3 3 3 3 

where u . > x , V . , * . • 

3- 3D 

is made, difficult by the need to solve a family of 
linear programming problems to find the ^^j^'s. 

Recently > the ^^uthor 'investigated further the use 
of L (x) as a. linear under approximation for ranking 
extreme jsoints for the fixed charge transportation 
problem as suggested by Cabot [4] . In this 'work, 

' Ije used a ranking procedure specifically developed 
for trauisportation ^polytopes . [19] . The nine prob- . 
lems developed by Gray [9] were used, as bench marks 
for comparison with other procedures. In Table. 1 

* below, we show the results from this computational . 
testing. In this we have shown, for each problem 
the size (MxN) ^ the relative sizes of the fixed and 
continuous portions of optimality (F*/C*) , •the numr— 
ber of extreme points ranked to prove optimality, 
the solution time on the UNIVAC J. 110, .the solution 
time for Kennington's [13] branch-and-bound proce- 
dure on the CDC Cyber 70, and Gray's original times 
on the Burroughs . 5500. ^ ^ ^ 

• As we can see , only on problems 1 , 3 , and 8 are our 
times competitive with those of Kenniiigton or Gray.* 
These problems . iare the. ones with large* continuous . 
portions/ and this is to be expected. This proce- 
dure does seem, to be better than the original rank- 
ing algorithm suggested by Murty in that at least 
we were able to solve all of the problems , while 
McKeown could only solve 1, 3, and 8 using Murty 's 
algorithm [201. ' • ' ■ ' 

Table 2 below compares extreme point ranking re-, 
suits using the original Murty algorithm and the* 
Balinski Approximation. Both of these were run by 
the author using the previously discussed ranking 
' procedure and are for the . same nine .problems in ■ 



. •* ^ ' ^ ' TABLE 1 

COMPOTATIONAL ■ RESULTS USINCf. BALINSXI'S APPROXIMATION 
Tt> SOLVE FCTP'S ' . ' ■ / 





.Size.: 




: Nurobei; 


*1110 


5590 


Cyber 


" Problem 


(MXN) 


\i F»/C» 


Ranked 


Time 


Time • ^ 
7.7. 


• *72 Time 


1. 


. 3X4 


, ■ '- ■ 

59/270- ' 


3 ' 


0.09^ 


.08 • 


2 




'95/108 • 


732 


. 53.36 


. 32.6 ' 


1.35 


3 


4x6. 


52/1947 




' .0.15* 


26.3 


.08 ^ 


4 


4x8 ' 


108/164 


- - - 756 


124.0 . 


U71.4 


. ,i-,io 


: 5 . 


5x7 


117/126 


*** 




- 263.8 


4,93 . 


• 6 ■ 


• 5x7 


- 173/143,. 


■15'4' 


18.70 


.149.9 


0.99 


7 


. 5x7 


1472/166 ' 






97.0 


1,48 


8 


* 5x7- 


59/2230 


\ 2 


; ;20- 


3?62.8 - 


0.11 


9' 


63^a . 


120/194.- 






1510.1 ' 


14.52' 



/ (All time in seconds) 
*** - procedure aborted due to storage overflow 



■ TABLE 2 

COMPARISON OF MTOTY'S APPROACH TO 
BALINSKI'S APPROXIMATldk . 







< *. 






Size 


fjvmber of Extreme 


Points Kmked 


Problem 


MxN 


Miirty 


Balinski 


1 


3x4 


29/ 


• 3 - 


2 


4x6 


*** 


732 


3 


4x6 


2.* ' ■ 


' '2' 


4 


4x8 


*** ' - 




5 


5x7 




*** ' 


6 - 


5x7 


*** ^ . 


154 


7" 


5x7 - 




*** 




S»7 






9 


.£x8 




*** 



- stopage overflow 



In this we note, that* the BalinsJci Approximation is 
quite a- bit^more efficient than the Murty approcich 
both irf^'number of points ranked for those problems 
that' were solved, as well as for number of prob- ' 
lems actually solved. 

Ill- Concave Quadratic Programming. Problems 



If we define 



T 
X D 



s.t. xeS ^ . 

then vie may say . that the optimal soliition to P 
will "occur iat an extreme joint "of S under any- 8ne 
of the following conditions , as. enumerated by Cabot 
and Francis £3]; 

1* The matrix D is negative definite or 
negative semide'finite. , 
■ 2 » Problem P-. ^is a quadratic programming , 
formulation of a problem occurring in 
■ bimatrix games.*. * 
• . ^- ^oblem P is a!=^ quadratic assignment 

problem. ^ * ^ ■ " 

These were not- meant to be inclusive of all con- 
ditions under which the optimal solution of P 



occurred at an extreme point, but rather, were ex- 
amples of such conditions. We will consider con-' 

i'dition .3 in a subs'equent section,. and there does 
not seem to have been any computational' experience 

r/in solving bimatrix games via extreme point rank- 
ing- As a result of this, we will restrict our " 
att'eijtion to problems 'fi-tting condition 1,-^i.e., 
D negative semi-definite *or negative definite. 

For this case; Cabot ^d Francis, show^tliat if we 
solve, the 'family of liilear problems, r\helaw for 
each column of D, d,, then the mlnimum^value of 
the objective function, u,, ca^i -be used to 'deter- 
mine a linear' under appro3timationV'L<x) . ^ 



/ 



Min 



3 



S.t, -X.S 
Now, using u.^, we write P 
approxinatioS problem as:^ 

n ' 

Min L(x)' » Z CC. 



£is the linear tjader ' 



j-1 



X. 

3 



S.t. XCS. 



• A major drawback in rising this approach is th^ . 
necessity of coii5>uting^ the u. valxJfes. One special 

• case where this is not a proSl^ is the quadratic'/ 
transportation problem P as- formulated below: ^ ' 

. .. * 'm N 

Z ' Z (C. .X. . + d_x. .) ^"^ . ^ ' 
ij 0.3 . - - ... 



Min 



N 

Z (C. .X. . 
i«l j-1 
M 

S-t, S X,, » A^ 
i=l 
N 

^ Z x: 



i3 



3-3 



1 



i^l , . . . fK 



and 



X. . > 0 

i3 - 
d^ . < 0. 

i3 - 



for all i; j 



This- problem weis formulated ets a way of modeling 
the .Mrginally decxeeising cost characteristics of 
the private motor freight industry [24].. As Cabot 
and Francis note, ia = d fminCB.,A.)l and we'may 
easily formulate thi^obje§Sive function for (P ) 
for the quadratic transportation problem. 
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It should be noted that in the case of where one is 
attempting to model the decreasing marginal cdst- 
charatteriatics over various routes by use of Pj^» 
it would be expected that the quadratic portion ^ 
will be small relative the continuous {portion. ■ 
In fact, we would expect that C,x, + d. ,x" would 
be positive non^decreasing over ^ the ent2r5 range ^ 
of rvalues of x , or C >^ |2d..u^.I for each io. 
We would also expect that the erfeStiveness of 
(P ) for solving the problem by extreme poinf^ 
raSxing would increase as C, , increases relative 
to d. .U, .r 

ID 13 ' • ^ 

To test ^ this, we used- two sets -of five randomly 
generated >^uadr a tic transportation problems ranq-> 
ing in size from 3x4 to Sx7- In each case we used 
the Chernikova algorithm as modified by McXeown 
and RubinrE19J' to h£mdle degenerate problems. The 
continuous cost ranges are shown below: 



Case I u 



Case II: 



2u. ;d. . 

^3 ID 



500 



< 5000 



2u . .d. . + 500 <<:..< 9999 
ID ID iD - 

In all problem sets, the quadratic costs were be- 
tween 0 and 10. In problem 5, these costs were 
between 0 and 20. The results -of this testing are 
shown in Tabl e 3 . ' 

From these resful'ts, we note some differences in 
nan time we move from Case I problems to Case 
'II. This Is felt to be due to the increas^ing dom-- 
inance <;jf continuous 'costs* over ^quadra tic costs in 
the latter cdse- We *also note an increeise in both 
parameCers afe we move to large problems for Case 
I. This iis to be expected due to increasing prob- 
lem sise. This does not happen in Case II since (5, 
vezry few solutions need be. ranked in this case. 

Problem Set '5 is the same as Problem Set 2 except 
that the remge of quadratic dosts has been^ in^ 
creased. Here we note a great increase in solu- 
tion parameter values in Set .5 O^rea. Set 2 for Case 
I. This is due to the decrease in aominance of 
continuous costs over quadratic costs. 

' ■ ■ - r , ■ ' ' y.. . . . 

TABLE 3. 13 

SOLUTIOM TIMES AND MUMBEI^^^aP^SXTREJ^E POINTS 
RANKED FOR VA|fc±OUS CONTINUOUS COST RANGES 



Problem 
Set 


Size • 
(MxN) 


d. . 
^D 
Range 


• Case- 1 
■ Tine 


V. 

tase JJ 
' Time 


1 ' 


3x4 


0-10 - 


* .09 


.08 




4x6 


0-10 


• .34 


.31 




.4x8 


0-10" 


.86 


.84 




5x7 


0-10 


,1.78 


,35 


> *• 5 


4x6 


0-20 


<1.13 


.41 



In terms of results, these problems tend to be 
jmuch more' promising than any of the others dis- 
.:cussed in this paper. If further research demon- 
.strates , that, for problems of the n^tur^ discussed 
by Oi and Hurter, ^the costs* are indeed marginally 
decreasing with dominant continuous costs ^ t^ien 
thi^ ranking procedure may be useful for solving 
such transportation problems. " - - 

IV. The guadratic Assignmenrt Problem 

One £Lrea of mathematical programming where it also 



, has long been known that , the soli^tion would occur 
at an extreme point is that of variations of the 
assignment problem, one such variation^ the well ' 
known traveling salesman problem vi'll be disqussed 
in a later section. 'Here we will discuss the as- 
signment problem whcrc^there are interactions be- 
tween the assignments^?]- "^his is commonly. re- 
ferred to as the quadrap'ic assignment problem and 



may bo fonnulated as P. 



Min 



S. t. 



, n ■ n 
E I C. .x^ . 

i-lO-l "^ '^^ 
n 

i-1 

n 

Z X 



AQ 



beiow: 
n n n n* 

I Z Z Z 
i o j q 



K, . X, .X " 

iDPq iD pq 



"ik 



ik 



k .1,, 



,n. 



■i»l,.*..,n. 



ik 



where K. 
and p 



is the co'sft of an assignmeijt of i to j 



That the solution of P would occur at an extreme 
pb^int of ,the cons-^a,"^? set was f ir&t .noted by 
Gilmore [7]. A^ lineeu: under approximation to 
was suggested by >X*awler [14^3 and may be" formulated 
as follows • .denote for each i, j, a minor -of 
,K . . as K ^ and denote ;the value of 'the solii- 
ti^S^of the Cn-l)x(n-l) assignment *pr obi em corre- ' 
rfepo;iding to ^aqh minor as Z . We also define a 
4:ost f . . ■ = Z + C. .- The solution to the as- " , 
signmen J- problem for Jhe matrix F f {f. .} is a 
lower, bound on "fhe'solution to *p-r . As'^a result, 
we may use F, the assignment proD-iem with cost co- 
efficients {2".'.}, a3 a linear under approximation ^ 
•to P ^or ex^eme ppint* ranking purposes. This 
is ail analogous procedure to that discussed for 
finding a linear approximation to quadratic pro- 
gramming problems with concave objective function 
discussed^ earlier [3] . ' 

In some prob34emSf we may define 



iDpq" ' 



t. d. 
iP DP' 



This is true in the Koopmans^Beckman single com-' 
modity problem. 

An extreme point rcinking procedure for solving 
this and other non- convex quadratic minimization 
problemis was suggested by Cabot and Francis [3] 
using ,the linear under approximation F. This pro— 
cedure is. essentially that discussed earl'ier for 
this type of linear under approximation in that 
•extreme points. of S are ranked until-^a value x^ is 
found such that i, where x* is the op- 

timal solution. " . ■ 

Using this approach to the quadratic assignment 
problem, Fluharty [6] wtote a master's thesis on 
the problem. In that wprk, he tested various prob- 
lems generated by' previous researchers who had 
worked on the quadratic assignment problem- Using 
a procedure suggested by Murty [21] for deteraiining 
adjacent extreme points for assignment- problems, 
he tested problems for 4v< ri £ 12 with mixed re- 
s^ults- Only in those cases where the continuous 
values, i.e., {C..}, were large as compared to the^ 
quadratic values^^as he consistently assured of 
not having to enumerate all nl extreme points to 
reach a solution. / For a proJ^lem with n = 12< 
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< C. . < 99, and 0 < t. \d. 



< 10, the procedure 
* However , when the 
max C. . was increased to 99^, the problem i^as 
sol ved^y. ranking only 100 extreme- points. ' in' 



overrail availal^le' core s€orafe 



Table 4^ 


>. we see the results .of Fluharty * ; 


s work. 




- 


. TABLE 4 








SUI1MARY 


OF FLUHARTY 


'S RESULTS 










t. ,d. 


Solutions 


• V 


- . ■ 




ip iq ' 


Problem 


SizeCN) 


Range 


Range . 


. Ranked 


1" 


- . ■ 4 • 


• 

(0,0) . 


(0,20) 


20 


2 


5 


' <0,0) 


(0,5) 


11 


3 . 


5 


(1,5) 


(0,5)- 




4 


6 


{0,0) 


(0,10) • 


. ^ 720 . 


* .5 


6 


(0,9) 


(o;io) 


. 202 




' . .6 


(0,99) 


.(0,10) 


' 10 


• 7 


7- 


(0'',99) 


(0,9) 


135". 


8 ' 


* • 7- 


. (0,0) 


(0,10) -, . 


5040- ■ 


9 


• - "7 


(0,99) 


(0,10) 


170 ^ ; 


lb,' 


7 


(0,9) 


(0,10) 


1848 


• XI ■ • 


7 . 


(0,0) 


(0,99) 


3201 


. 12 


7 • 


(0,99) 


(0,99) 


3212 


13 


.8 ■ 


- (0,0) ■ 


(0,10) 




14 . . 


8 ■ 


■ (0,9) 


(0,10)./. 




15 


8 . 


(0,99) 


(0,10) 




16 


10 


• (0,99) 


(0,9) - ^ 




17 ' 


10 ■. 


(0,999) 


<0,9) 


188 


18 


12 


(0,0) 


(D,10) 


*** ' 


19 


12 


(0,99) 


(0,10) 


*** 


20 ^ . 


12 


(0,999) 


(0,10) 


100 



♦♦♦-Storage overflow^ 

• Once again the importance of relative size of lin- 
ear vs. nonlinear portions of the objective func- . 
tion becomes evident for the use of extreme point 

^ procedures . . 

V. Traveling Salesmaui Problem 

- . - ' - ■ . ■ ■ . 

With aJ.1 the . work on using extreme" point ranking 
algorithms to .solve vaurious problems with' linear 
constraiints, it might be surprising"" to find that 
very little work- has been done, on the -traveling 
salesman problem. It can be easily seen 'that the 
. optimal traveling"* salesman tour can be found by 
ranking the solutions to the assignment problem ^ 
until the tour is found. However.,' our research 
has shown that only some very e.airly work by Murty 
"* "and Karel (23 j has been documented. However ^ 
they opted to move onto their branch; and bound al- 
gorithm [15] instead of continuing work on extreme : 
point ranking methods.^ 

In this early paper ,^ a method for ranking the as- ' 
signmenta^as presented together with a procedure 
■ to avoid gjinerating adjacent assignments that -con- 
t2Lined. subtburs. Using 'this procedure, a ten city 
randomly generated problem was solved "...by Hand; 
and the , time taken was about half an hear." Also, 

•a 20 city symmetric problem was sol'^ed. This . 
" — involved <lthe solving of 10 different assign- 
ment problems of sizes ranging from 16 to 20. On 

. the 'Burroughs 220 computer,..., this took about 10 
minutes in all.^ [20] As mentioned earlier, tbis 

. worJ^, • though not ptiblished, resulted, in a branch 
and bound etlg<yrithm. ^ Also, the method for ranking 
the •assignments presented in this paper wasr later 



published [21] . • . \ * ... 

^ One" other bit of unpublished work in this area ' 
. 7 . came to .light /f5com Sweeney and Williams [27]. It 
,^Pij,Twas stated by ' them that for a problem on the order 
^gr^/: 'tt 40 ci^es,. approximately 5000 assignments were^* ' 
•yiV*";. ranked without a- tour being found. 

. * One 'fxirther comment on the traveling salesman 
prob'lem concerns another paper by Murty C22}vQn 
' the toiirs of - the traveling salesman problem. Ii?^ 
that paper, he proves that all solutions that are 
tours can be found ^y determining the vertices ad- 
jacent to the diagonal assignment solution, i.e. , 
X.. =1, i = 1, ...,.N. However, due to the large 
nimiber assignments adjacent to acay other assign- 
ment, xinless some efficient procedure can be 
found for generating these tours ^in a non-increas- 
ing costwise, manner , this does not appear promis- 
ing- ~ . . 

VI . Conclusions and Directions for Future Research 

. From duTj look at the use of extreme point ranking 
procedures to solve problems with concave objec- 
tive fimctions and linear constraint sets, it 
would seem safe to say that the efficiency of this 
procedure .is extremely dependent upon the ob^ec-^ 
tive function. In those problems 'where the objec- 
tive function was approximately linear, the proce- 
dure could be expected to be very efficient. . How^ 
ever, as the non-linearity of the objective func— 
tion increases, the efficiency of the procedure 
decreases markedly. 

SeVeral £ureas of future research into the use of 
this^ procedure present themselves. One woxol'd be 
to combine the use of the Tui cut with methods for 
efficiently determining adjacent vertices . in the 
presence of degeneracy. Another possibly fertile 
area of research would be tq look more closely at 
the general linear -fixed charge problem with 
greater-than constraints* Finally, resefifirch might 
be profitable in implementing Cabot and Francis' 
approach to more general ^ concave quadratic pro- 
gramming problems. 
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^ - ABSTRACT " 

During the early 1970' s,. highly efficient* ^ 
i' special purpose computer codes- were developed for - 

solving capacitated .transshipment proSlems based , 
' on primal- extreme:; point algorithnis^ 'Computational ■ 
^ comparisons of ■-these" codea with the best noflr- 

extreme point codes designed for the same class of 
problems , indicated, that the primal extrei^e point- . 
based cpdes were subs tan tiilly. superior both in ■ . ^ 
". terms - of computer . time and* ;nemory -requirements on 
. all typesL.of network problems^ ♦Jore recently, 
"specialized non-extreme point codes have. been 
developed for. uncapacitated bipartite problems — 
notably assignme^it and semi-assignment problems. 
Comparisons- of . these prq^jiem specific non-extreme 
point codes with; the>fearlier/ general-purpose * " ■ 
extreme point codes casts doubt on ;the earlier 
computational concJUsions.. . Coiisequehtly, the 
; - purposes -of this ps^p^- are-to develop a new ex-.V 
, .;trerae.' point algoritl^which is"" specifically . ■ 

-designed to. ,take: a d\i^t age of bipartite .b^ .. 
- - flows, and degeneracy -aspecCs of assigjunent an«f 

■ .^emi-assignment problems and, f urthier,.'tp -.'conduct ' 
'■'an unbiased comparis'on of the . ai t e ma tive al go-. ^ 

■ -rlthmic approaches for solving assignment and semi- 
assigmnent .profilems- . . - ; 

1.;. INTRODUCTION \ ' 

The semi-assignment .problem is a bipartite- 
.network problem whose supply constraints are the 
same as those of an assignment problem and whose 
demand constraints are the, same as, those of a 
..transportation problem, or vice versa. In the 
midst of the dramatic advances [1,2,5,11,14]- in- \\ 
network solution technology since i^69^- this 
important member of the network family called the 
. . semi-assignment problem has- received scant atten*- 

■ tion. Falling midway between the classical assign- 
ment: problem and the classical 'transportation in . 
^ts-' generality,: it. was bypassed alike by . those . . . 
who. studied the ultra-specialized assignment, 
structures' and those whof'studicd- the idore general 
'bipartite transportation structures.' 

.^Ihe'neglect of ^the semi-assignment problem — 
is especially ironic in view of the fact that it 
occupies one of the singularly important niches in - 
the network hierarchy. The "assignment half" 
captures the ub;iquitous multipile choice structures 
of capital budgeting and planning problems and the.c.. , . 
special ordered set constraints of mixed integer 



ERIC 



223 



and combinatorial programming. The "transporta- 
tion half" captures arbitrary upper and lower ' 
.bounds on disjoint sums.. of variables, and there- 
fore can provide. valid relaxations for any mixed 
-integer program >?ith imbedded-/multiple ch^ and 
special ordered set structures.-v>- Still mbre'di- 
rectly,. the jsemi-assignment structure appears in - 
large scale scheduling and planning;;problems from . 
real wtfj^d settings. For example, applications of^ 
manpower t planning . (assigning personnel to jobs), V" 
scheduling (assigning aircraft to routes, trucks 
• to routes, freight to transports, etc.) , projett 
planning (assigning project components or. sub- 
assemblies co^tasks, over time), and a varie^ of ; 
other practical problems in , planning logistics . ^ * .■ 
contain- embedded semi-assignment problems. ■ 

Consequently, the purposes pfs this paper ^are 
tj>>develop a new ..extreffle.-poin't'' algorithm (called^- 
^, the-raltefnating'*basis (AB) algorithm) which is: -"■ 
■ specif ically designed' to take advantage :bi-. ■ ; . . 
partite , boolean, flow, and , deg^eracy aspects^ of • .* 
. assignment and' semi-ass ignnen't problems and,; fur--. ^ '': 

ther,' • to concfudt -a comparison, of the alt e^ ■ " 

- algorithmic approaches, using codes designed .for - 
. solving these problems. ' . ; 

: Computational testing has -.shown that approxi- 
-mately "90 percent of the pivots within special ; 
purpose -primal simplex-based .algorithms [2,11,13] 
are degenerate, for assignment and semi-assignment 
problems with more- than 1000 nodes. The primal • 
extreme point algorithm presented in the paper for 
solving semi-assignment problems jurhich both cir- 
cumvents and exploits degeneracy can be viewed as' 
-an extension of the algorithm presented in [4J and ■ 
a specialization , of the algorithm presented in [8].. 
^One of the principal features of this algorithm i^s 
a strong form of convergence that limits the num- 
ber of degenerate steps^ in a far more powerful 
■way .than achieved by "lexicographic improvement," 
as for example, in customary LP perturbation ^ 
schemes. ; ^. 

Each basis examined, by tMs algorithm ; is 
res trie ted to have a certain' topology , We' show* ■ ' 
that i'f a semi-,^ssignm'enc problem has an "optimal"' 
soljLition,. then an optimal solution can be found 
by considering only bases of this type. ' The major 
mathematirsS differences between the AB algorithm 
and the simplex method are (1) the rules of the ■ ' 
algorithm automatically (without search) assure 
that all bases have the special topological struc- 
ture, and. bypasses all^ other. bases normally given 
.consideration by the simplex method; (2) the algo- . 
rithm is finitely /convergent with Ajt reli^ance upon 



(sucn as 



■ "external" .techniques (sucii as lexicography" or 

perturbation); and (3) in certain cases non- 
^^egenerate basis exchanges may be recognized prior 
' to finding the representation of an incoming, arc. • * 
For. thdse reasons^ this .algorithm has several com-* 
piit^tional advantages over the highly efficient - 
special purpose simplex-based codes recently* devel- 
oped for solving network' problems, 

■The'AB algorithm also has unique computer 
•implementation properties. ' Specifically, the data 
-required to. represent its bases are subsCantially 
less- than that required for general simplex bases; 
thus, the computer memory required to store the . 

• basis data is less .ti>an that of special purpose* 
simplexTbased algorithms. The computational re- 
sults' in section 6 *• dramatically demonstrate the 
power and .efficiency of the AB algorithm over 
other algorithms "for solving assignment and semi- 
assignment* problems. . ^ 

; 2 . • BACKGROUND' MATERIAL " - / . . 

Anm-x n semira^signroent problem may be de- . 
fined . as : ' . " 



Minimize ' * I 

(io)£A 
subject* tcr: ' 

je{j:(l,j)£:A}; 



c - b \ iel -:{l^2,...,m} 
1* jcJ-.- ■{Iv2,...,n} , 



of one and the other m - 1 arcs have a" basic flow 
value of zero. Therefore each basic- solution is* 
highly degenerate (i.e.,, contains a- large number* 
of zero • flows).. This often causes the simplex - - 
method to examine several alternative bases for » 
the same extreme point before moving to an adja- 
f cent extreme point • • ' 

In the graphical representation, approach", the 
bases- of the simplex method for. semi-assignment ^ 
-problems are normally kept. as rooted trees . / 

[5, 7, 12 > 14,19], Conceptually, the root node may beJ^ 
thought^ of as the highest node in the" tree witlT* 
all of the other nodes . hanging below "it on direct- 
ed paths leading downward from the root. .Those 
.. nodes in the unique path from any given node i" to 
the rbot^ are called the ancestors of node i, and.. . 
•the iiumediate ancestor of node i^ is called its'. 
predecessor. , ■ 

Figure 1 illustrates a, rooted basis tree> the 
». -predecessors of the nodes, and the basic flow 

values, for a 3 x 6 semi-assignment problem. 
> Notationally, 01, denotes the ith origin node and 
; Dj denotes the j th destination node. .. The number, 
beside each-link (arc) " in the -basis tree indicates 
the "flow on this . arc imparted, by the basic solu- 
tion. Predecessors of nodes are identified in the 
PREDECESSOR array. For example, as seen from*. this 
. array, the predecessor of/origin node 2 is d est ina- 
ction node 1. The root of'^ihie ; tre.e is node 01 and ^ 
. has Tiq; predecessor. ■ . . . • ' ^ ' - ■ ; " 



where It is called, the set of origin -nodes, J is 
called the. set of destination nodes, A ,is the. set 
^ of. adndssible arcs7 «'ci cij* is "the- co^t of ship- 
piCTg a unit -from' origin node i . to destination * 
:*aode j;- ■ ' ■" _ . . . 

. . . ^ The dual of . the semi-assignment problem may 
be stated, as: / ■" 



Maxipjize ' -Z ' 
iel 
subject to r 



'Vi 



'where " R,., 



-j^id K are called "the node potentials of 
the. orison and** destination nodes, respectively- 

All understanding of the results of this paper 
, relies on a^ familiarity with graphical interpreta- 
tions of the semi-assignment .probiem* and* how the 
primal simplex .method may be applied to this pro- 
blem. -While these ideas are- relatively direct, 
they, unfort.unately , are not succinctly itemized, 
in any -references and -will be summarized. in this 
section, for " completeness...... ' . "■■■'"'■j^. ' ' ■ . ' ' * 

The semi-assignment problem may. be rep re- " 
sented as a bipartite graph consisting of a set of- 
origin nodes with supplies; b^ and a set of- destina- 
tion no des~ with unit demands. / Directed arcs from, 
origin nodes to destination nodes accommodate the " 
transmission of flow and incur' a cost if flow 
existis. The objective is to determine a set of 
arc flows which satisfies the sufply and deiaand - 
requirements at minimum total costT^ 

The bases of the simplex -methdf^^f.or soJ?ving 
; an .m 'x n semi-assignment "problem correspond ' Co 
spanning trees with; m + n 1 arcs. Exactly n of 
the basic arcs have ah- associated basic flow , value 
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Figure 1 — Roo ced Basis Tree . 
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' It is important to note the direction of. the links 
. In Figure 1 correspond^.to the orientation induced 

by the predecessor ordering and do not necessarily 
.. correspond to the direction of the basis arcs in 
the semi-assignment, problem. However, the direc-*/ 
tion of the basic arcs are .kn.dwn f romi; the bi-r ^ 
partite property of the semi- assignment problem;. 
i.e.» all problem arcs lead from* origin- nodes, to 
destination nodes. . . ..^ 

In subsequent -sections the term O-D IinH and 
X) llz2k will be used to refer to links in a'^ " 
rooted basis tree that .are directed; from an o-rigin 
node Co a destination node and vice versa » ^accord- 
ing to the orientation imparted to the basic '^rcs 
by the predecessor indexing. I^or example, in 
Figure 1, 02-D4 is an 0-D link .while Dl-02 is a 
D-0 link. Additionally^ basic ajrcs. with^a flow of 
• one or zero will be 'referred to as i-iinfcs and 
. respectively. ^ 

The. fundamental pivot step -of the simplex, 
method will, now be briefly reviewed in xhe graphir 
cal setting. Assume that a feasible starting 
basis has been determined and is ^represented as a 
rooted trtfe. To evaluate -the nonbasic .arcs to 

• determine whether any of them. "price out" profit- - 
ably, and therefore are candidates to enter the 
basis, it is necessary to determine values for the 

,.dual variables iel,: and Kj. jeJ, which satisfy' ' 
complementary slac^ess; i.e. , which yield^ 
f.^i each basic arc. ■ 

•There is a- unique dual " variable associated 
.wfth each node in the basis tree. For this reason 
\the diial .variables' — or their values— are dfte^ 
referred to asiaode potentials. Because of 'redun- 
dancy in' the. dSBfening equations of the semi- ^ 
. assignment problem (and in network problems gen- 
erally), one node potential may be; specif ied ' 
. ; arbitrarily. . The "root node is cuistomarily select- 
ed for this . porpose and assigned a potential of 
zero,., whereupon the potentials^ of the other nodes • 
are immediately determined in a cascading fashion 
by moving. down the tree and identifying the value 
-for, each node from its predecessor using the 
equation % Kj - c±y Highly efficient labeling 
.procedures for traversing the tree to initialize 
and update these. node potential values are des- 
\cribed in [5.12,14]. 

• •"A feasible basic solution is optimal when all 
-nonbasic arcs satisfy the dual feasibility condi- 
tion + K j < Cij. If the solution is not optir 
mal, then an arc whose dual constraint is violated 
(i.e., for which. -f Kj > Cij)'is selected to 
enter the basis. The arc to leave the basis is 
determinedrbyr (1) finding the unique path in 
the basils tree, called the basis equivalent path, 

• which connects the two nodes of the entering -arc,'' 
.and - (2) isolating a . blocking arc- in th-is'^path 

whose flow goes to zero ahead of (or at least as 
soon .as) any .others as a result of increasing the 
flow on .the entering arc. In the basis equiva- 
lent path, all arcs an even number of links away 
from th^ entering arc are called even. arcs, and / 
all arcs an odd number, of links away are callecl 
odd arcs., An. increase in the flow of the incoming ' 
arc causes a corresponding increase in the flow * 
of all -gyen arcs .'and a corresponding decrease in 
the flow of all odd arcs. * Thus, if. an odd arc 
already has a 0 flow, then such ^an arc qualifies 
as a blocking arc and the incoming' arc cannot be • 
assigned a positive flow. ; . 



.To illiistrate^ assume that the starting basis 
is the' one given in Figure 'l and the entering arc 
is (03, D4^. The. basis equivalent- P5ith for (03, D4) 
is. D4-O2-D1-O1-D3-03- (Note -that this p^th can>e 
easily detemiined by tracing the chain of prede- 
cessors of 03 and DA to their point of rbnterS'ec— 
tiqn [5,10,12],) As flow is increased on the 
enteringi^arc,^ the " flow, on the odd arc. (01, Dl) must 
be decreased. Since its flow is "already ^ero, 
(01;D1) qualifies as a blocking arc so that when 
/ arc (03, D4) is brought into the basis,* arc (01, Dl) 
must be dropped. ^ (There "are .no. other blocking 
. arcs In this case.) In -addition,' the pivot (A 
basis exchange) is degenerate since no flow change 
occurs." ^ \ : ^ 

Once, the entering and leaving arcs are known, 
♦ the basis exchange is completed simply by updating 
. the flow values on the basis equivalent- path and • 
determining 'Jiew node poientials for the new basis. 
. ■ tree. . . I . , 

Only^a subset of the node potentials changed' 
during a pivot and these can be updated rather 

■ than determined from 'scratch* This fact, will play 
a crucial role in proving convergence of the algo- 

■ . rithm to be vdeveloped. 

^ . .To update the node'potentials,^ assume that 
^ the nonbasic arc (p,q) is to enter into. the basis 
and the basic arc (r,s) .^is to leave the basis* ^If 
^^<^(r>"s) is deleted from the basis (before adding 

■ ■^^^^V'>^)). :t:wo subtrees a each con- ^ 
taining .one of tl^e two nodes the- incoming arc . 
(p,q5« let K.denotie -the subtree which does, not " 

'contain the root 'Tio'de of the full basis. The no"de 
potentials for the new basis may be obtained [12] ." ■ 
by updating 'only ttiose potentlals-of the nodes in - 
. K, as follows^^ If p is in K, subtract . - 
^■*- Rp^ ^ " ^Pq ^' ^ the potential of . each.;'" 

; origin node in iC and add 6. to ' the potential of' 
each destination node «in K. Otherwise, q is fn -K 
■and. -6 is used in the above operations^ ^> 

ALTERNATING PATH BASIS DEFINITION AND ^ 
■ PROPERTIES ■ 

. The new alternating basis (Ab) Algorithm for " . 
.semi-assignment problems developed in this^section .-' 
-is similar.-to the. primal sitnplex method-as .des- * 
jg^ibed' above. Its major mathematical distinction 
is . that it does n6t'consii.deV all feasible bases 
to be candidates for progress3fng to an optimal 
' l>asis.. That is, the. simplex method allows a' 
feasible spanning tree of any structure whatsoever ■ 
to be included in 'the set of those that are 
eligible for consideration as "improving bases" 
along the path to optimality.. However, it will be 
shovjn that if a seml-asslgriment problem has .^n 
; optimal solution then it also "has an optimal 
J solution with a unique basis tree structure, ' 
dubbed the alternating -path .(AP) structure. ■- 
Furthermore, it will be shown that it.is^ possible 
to restrict attention at each step to bases with 
this structure.. -In particular, the AB algprithm 
. is a procedure., designed to exploit the properties 
'of the AP basis structure in a manner that siib-, 
stantially reduces the impact of degeneracy, the 
number of arithmetic, operations, and the data 
^^storage locations required to. solve the semi^- 
asisignment problem. 

Definition : A rooted basis tree for a semi-, 
assignment problem is an alternating pat A (AP) ' 
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2^si s if : . 

. 1." ■ The root node is an origin node. .. ■ 
2. All 1-links are. links.. 
S. All O-lic^. are D-0 links." 

An example of an AP^asis is shown, in Figure 2. 

. The "alternating **pach" designation is applied 
because every path from a node to any ancestor 
node In the tree, or vice versa, is an alternating 
path of 1-iinks .and 0-links. Our attention will 
chiefly focus on paths from nodes to their ances^ 
tors (as would "be traced along ,a succession of ^ 
predecessors). A path that begins at an origin 
node and ends at an ancestor destination node : 
will be called "0-AP" because 'it begins and ends 
with a 0-link* Similarily, a path that begins at 
a destination node and ends at an ancestor origin 
node will be called "l-AP" because it begins arid' 
ends, with a 1-link. 




Figure 2— An AP basis for a 3 ^ 6 semi-- 
assignment ^problem ^ 

Remark 1 : The 1-links ".of any feasible semi- 
assignment solution can be augmented by 0-links 
■ to create an AP basis (e.g., by adding 0-llnks 
• from destination nodes to origin- nodes in' any ^ 
fashion , so that ^ every origin node except the root 
node' has. exactly one en tearing 0-1 ink.) Note if 
the arcs corresponding to the addjed 0-links do not 
exist in the- particular 'semi-assignment problem 
then a. large (big M) cost is assigned such links. 

Remark 2 : There are many semi-assignment bases 
^. for a- given feasible solution that are not AP 

bases. (For example, any basis that has more than 
.one 0-llnk incident to an origin node is not an 

AP basis regardless . of the origin, node chosen as 

th^ - root. Figure 1 is an.exampl^pf " such a basis.) 

Remark 3 : An artificially feasible AP basis may 
always be constructed for an m .x n- semi-assignment 
problem by assuming that: arcs exist from each ori- . 
' gin node to all destination -nodes where the no n-' 
admissible (artificial) arcs have a ''big M"' cost. 
: the procedure is as follows. 

. Initially set J* - 0, B - 0, and i - 1. Go 
to step \ 



1. Let r. be a 'destination node such that 
^±T tain c |. " ■ 

Set B = {(i,r) }UB,"Xir ^.1, J* - J*tf{r}, 
and bi ■ bi - 1. ' • • ^ • 

2. If h± ^ 0 go to step 1,' Otherwise, go to 
■ ' step 3^ -■• • ■ 

3. ,If i 1^ mp. set i » i + 1. and go to step 1* 
Otherwise set i 2, J* « 0 and go to 
step 4. • . . • 

" 4. Let r. be a destination node such that 
c^^ » min c^j and (i,r)^B 

' jeJ - J* 

Set J' « J'U {j:(i,j)£B}, B - {(i,r)}UB, 
■ ■ and Xir "0 "* ^. 

5. If 1 m, set i " i +1 and go to step 4. 
' Otherwise go to step 6. 

6. Using B, create a spanning tree rooted 
at node 1, The resulting spanning tree 
will be an AP basis, y ' 



Ptoof : 

*ThG remark follows by construction 
•Definition 



Relative to any AP basis, a nonbasic 
arc is called a' downward arc if it connects a 
destination node to an ancestor origin node, an 
upward arc if it connects an origin node to an 
ancestor destination node. . An arc. that connects 
an origin node "and a destination node that do not.' 
•:- have either of these ancestral relationships is 
■ called a cross arc. ' (Note that these are the. only 

.three 'possibilities for a nonbasic arc in a bi- 
, partite network.). ' ' 

The next two 'remarks point out some important' 
. properties .that can be exploited when applying 
the simplex method to an AP basis. 

Remark. 4 : When the simplex method is applied* to 
an ;AP basis,. a pivo£ is 'nondegenerate if and. only 
if the entering nonbasic arc is a downward arc. 

\. ' Proof: ■ . ^ •. 

«r " . . . 

The remark relies on the fact >that a xion-. y 
degenerate pivot causes the flows ;6n the bi^is 
equivalent path to decrease and increase in^ 
- strictly alternating fashion to the odd and^even 
links. The "if" part of ^the remark' then follows 
by. observing that a downward arc is 1-AP. The . 
"only if" part of the l^emark fallows from two ■ 
observations, first that an upward arc is O-AP-, 
and second that a CMss arc has a 0-l.ink above 
the origin, node incident to the entering -arc (and 
this arc is contained in the basis equivalent 
path adjacent to one of the 'nodes of the "entering 
arc) . , . ' 

Remark ' 5 : When the simplex method is applied to 
an AP basis, 'the." pivot can be carried out to give- 
a new AP basis for. any entering nonbasic arc 
simply by dropping tiae unique link in the basis., 
equivalent path attached to. the origin node of 
the entering aire."* 

Proof: , , 

The remark follows by observing that an AP 
basis results if a rooted tree is constructed 
"with its root node equal to the root node of the 
old AP basis.' ■ 
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Alternating Basis CAB) Algorithm 

pa the basis of .the "preceding remarks, .the 
rules of the AB algorithm can be stated" in an 
extremely simple *jfashion. ' - : . " - . 

1. Select any feasible A? .basis f otf the 
semi-ass igament problem (e.g..,. using 
Remark 3). 

2. SuCcess^ively apply the simplex pivot 
step keeping tjie root no<fe fixed and 
picking the link to leave- according to 

. Remark 5. 

. By means of these rules, the foregoing ob- 
servations imply that' the AB algorithm, will 'pro- ■ 
. ceed through a sequence of AP bases, bypassing all 
other basis structures. Further, , these remarks 
show that a "next" AP basis is always accessible 
to a given AP basis, so that the method will not 
be^compelled to stop prematurely without being 
able to carry out a pivot before the optimality 
(dual feasibility) criteria are satisfied. The 
"issue to be resolved then, is whether the method 
may progress through a closed circle of AP bases 
without breaking out, and thus, fail to converge. 
It will be shown that this cannot happen, and 
that, in fact, the AB algorithm is finitely con- 
verging without any reliance upon "external" 
'techniques such as perturbation, as in the 
ordinary simplex method, /^reover it will be 
shown that the form of convergence of" the AB 
algorithm has a particularly strong character, 
in which origin node potentials and destination 
node potentials, each change in a uniform direc- 
tion throughout any sequence of , degenerate pivots.- 

These results' do not require any res trie- 
tions on the- choice of the incoming variable. ' ' ' 
For -example. It is not necessary to ctai through 
pivot possibilities- in an attest .to- find de- , " . : 
generate pivot ' candidates . The following lemma- 
and .theorem validate these 'statements. 

Le mma : A basis exchange with the AB algorithm . 
gives rise- to *"a new AP basis ia which the new 
.node- potentials satisfy the following properties: . 

a) For a nondegenerate pivot : The changed ^ 
origin node potentials ;strlctly increase and the 
changed destination node potentials strictly 
de'crease;. * " J 

' W For a degenerate pivot : The changed ., 
origin node, potential^strictly decrease, and the . 
changed destination node poter^tials strictly 
increase. 

Proof : " . 

•r As already discussed, the node potential 
values that change may be restricted to those 
asisociated with the subtree K. By this pro- 
cedure, if subtree K contains the origin node 
of the entering arc then all the origin node- 
potentials in K are decreased and all destina- 
tion node potentials in K are increased. The 
reverse is true if the destination node of the 
entering arc is in subtree K. The lemma then 
follows from Remarks 4 and 5, which imply that 
subtree K always contains the destination node of 
the entering arc for a nondegenerate .pivot^'and. 
the origin node of .the entering arc for a degen- 
erate piVot. " . • . ' 

*. Our main result may be stated as follows; 



'. -^ Theorem : 

The AB algorithm will obtain an optimal solu- 
tion (or determine that the problem is Infeasible) 
in a finite number of pivots, regardless" of which 

■ dual infeasible arc is chosen to be the entering 
'arc, and without any reliance on perturbation or 

lexicographic orderings. 

Proof : 

If is sufficient to show that the number of 
degenerate pivots that occur between any two non- 
.. degenerate pivots must be finite. This follows" 

■ from the 'second half of the iemma. Note that the - 
node potential assigned to the root node never 
changes when the node potentials in subtree K are . 
updated. Thus given the constant node potential'- 
fof :the root, the other node potentials are 
uniquely determined 'for each successive basis 
(rega'rdless of the procedure .by which they are 
generated) , and the unifomi decrease of origin 
node potentials and the uniform increase of des- 
tination node potentials (for the potenti-als that 
change) implies "that no basis can ever repeat 
during an uninterrupted sequence of degenerate 
pivots. This completes the proof^ 

^ A. C0MPT7TATI0NAL CONSIDgtATIONS 

Some of the unique computational, features of 
the AB method include; • ' 

a) It explicitly bypasses all "non-AP" basis 
. solutions without requiring any imbedded search 
procedure or computational tests. ^ 
■ b) It allows degenerate pivots to be recog- 
' nized and performed without computing the repre- 
sentation of ..the entering arc. '.This can be accom^ 
;. plished by using the "cardinality function" of ' 
Srinivasan and Thompson- [19] which indicates the / 
, number of nodes. in *th6 subtree; of the; basis tree" 
below a: given node. . In particular, s^following" the 
labeling ideas. recently proposed .ln^/[5], .upward 
arcs and some cross arcs can; b^ detected simply by 
comparing the cardinality function values of the 
nodes associated with .the entering arc* That is, 
denote the cardinality.- function by f and: the 
■ enterinjg arc by (p,q). If f(p) < f (q) then arc - . 
(p,q) is either an upward arc or a cross arc. In ' 
either -case the pivot is: degianerate and no- flow . 
updating is required. Remark 5, furthermore, 
"directly specifies .the link to leave the basis.., 
Thus.' a degenerate pivot simply involves checking " 
the cardinality function, inserting and deleting 
the appropriate links, and updating: the node 
potential values. 

' c) Similar streamlining Can be achieved for 
all other pivots. . Specifically,' if f(q) < f(p) 
. then the- appropriate step is to find the first 
node z on the path from q' .to the' root node such, 
that f(2) > £(p>.; If z> p then-arc Cp,q) is a ' 
cross arc and thus .the pivot may be executed as 
before- * If 2 " then, the' arc (p,q) .is a downward 
. arc^and the pivot. is no'ndegenerate- . Note that it ' 
is only in the; case of - a nondegenerate pivot that 
the entire basis; equivalent path of the entering . . 
arc is traversed. ' Thus 'it. i-s* only in this case 
that, the complete representation of the entering 
arc is computed. This is, of course, .substantially 
different than for standard network methods^ ■' 

Steps b and c above can also be accomplished . . 
by using the "distance function" proposed by . 



■ Srinlvasan and Thompson [19]^ which indicates^ the . 
number of links- in the basis tree between a giv^n- 
node and the root node. In particular, • the dis- 
..tance . function may be used in place »of the 

• cardinality function by reversing all the. above, 
•inequalities. < * " • * ' • 

d) Flow values, on the basis links never have 
to* be checked to determine the type of pivot. In • 
fact, the structural property of an AP basis, 
whereby, all 1-links are 0-D links, makes it un- 
necessary to store or update any flow values. . 

e) All of the above computaticlnal features 
.may be .further enhanced by observing that it is 
n6t "nec^ssary to store and update a ."full" basis. 

* tree. That is, while the predecessors are Re- 
quired for each node,, the cardijnality. function 
values and the "thread" pointers (commonly used 
to traverse basis subtrees [5, 7, lA] for, node • 
potential updates) need be kept for only the 
origin nodes. Using this observation, the AB 
algorithm's basis data storage requirements are 

. roughly half that of the most efficient imple- 
mentation involving specializations of thesimplex 
method. Moreover, the compression may be used 

-•to greatly reduce the number of nodes traversed 
at each iteration when updating the potentials* 
By maintaining node potentials for only the ori- 
gins and saving tKe unit costs for the arcs 
represented by tte destinations and their prede- 
cessors, all data necessary for the pricing opera- 
tion is available. The destination node potential 
may be compuced as needed using the comple- 



mentary slackness relationship K 



This partial update of the node potentials was 
originally proposed and tested by Harris [15] for . 
.the simplex method as appl'ied to "rectangular 
transportation problems. * (This approach . has also 
been tested, more recently for transshipment net- 
works by Bradley; Brown, and Graves [7], who have 
confirmed Harris's findings of its practical ' 
merit.) ^Harris's proposal, however, differs from 
the above in that • the thread pointer and potential 
update operation is eliminated for those destina- 
tion nodes of the basis tree which have no 
descendencs. Consequently, Harris's subtrees are 
twice as Ia.rge as in our proposal and therefore 
involve twice the. updating effort. . In the AB 
algorithm, maintenance of only the thread pointers 
and .node potentials associated with the origins 
does not degrade the efficiency of other parts 
of xh^ algori.thm.' This! is due to the unique , 
.structural properties of the AP basis. 



. "code whose capabilities did not depend on the 

* unique characteristics of a particular computer 
(e.^!, word size, etc). The obvious advantage of 
this approa..ch is the, ease with which it enables , 
the code to be -tested- on different machines.; • 
Further, we used a* "manilla"' FORTRAN IV so "that 
♦ receding to fit differing, machine convent iorrs 
^ would be minimized. , Within these constraints, we 
tried^ to minimize our storage' requirements,, at the 
same time making sure the code could solve the 
."thoroughly general" semi-assignment problem. . 
The code uses the predecessor [10], thread. [1^], 
and distance [19] functions to. maintain and. '.update 

' the basis data. , , \ ^ 

A variety of start procedures could/ be used 
to find a starting AB basis. However,' due to 
severe time pressure, *we only implemented the 
start procedure of Remark' 3. 

An important factor influencing computational, 
efficiency is the basis change criterion. The. 

- relevant tradeoffs for the basis change criterion 
involve time consumed in searching for a new arc 
to -enter the basis and the number of pivots re- 
quired to' find- an optimal solution (time per 

■ pivpt versus total number of pivots). Computa- 
tional testing [2, 7, 11,^3, 18,-19] has shown that 
the correct pivot criterion can reduce solution . 

^ time by as much^as a factor of three. Unfor- 

■ tunately, we did not have time to test alternative- 
'. piyot criteria. The code simply *.uses the row 

most negative rule\. which was found to be the best 
in the studies [13,19] ror small problems. This-- , 
criterion scans the arcs of each origin until it 
encounters the first origin containing a dual 
infeasibility and then selects the arc of this • 
origin which violates dual feasibility by* the 
largest amount to enter the basis. . ' *• 

The program consists of a main program and 
three subroutines - The total time spent in each 
. . subroutine was (.recorded by calling a Real Time 
Clock (accurate to a hundredth of- a second)' upon 
entering and leaving that subroutine. A count 
was also made of -the number of nondegenerace and 
degenerate pivots performed. In the . following 
sect ion we discuss total solution time (exclusive 
of. input and output), the start time, the number 
of nondegenerate and degenerat^ pivots, the total 
pivot time, and the average pivot time (total 
pivot time divided by the number of pivots). This 
' code will henceforth be referred to . as SA-AB code, 
for semi-assignment AB algorithm code. ^ 



5 ; DEVELOPMENT OF THE AB COMPUTER ' COPE 
BY SUBROUTINE 



• The computer code: was written in FORTRAN IV, 
is an incore code',* and was initially tested using- 
. the run compiler on a CDC 6600 with a maximum 
memory of 130,000 words. In this; code, a semi- 
assignment problem with M, origins , N destinations, 
and A arcs, (without exploiting the word size -of 
the machine.) re<iuires- AM + 2N + 2A + 5000 words. 
It would be possible hy exploiting the fact that 
the . costs, node numbers and node potentials are. 
integer- valued, to store more . than one per word 
and in this- manner reduce these storage require-'* 
ments* How[ever, our purpose, was to develop. a 



6 . COMPUTATIONAL COMPARISON AND CODE 
REQUIREMENTS ' 

6,1 Computational Comparison of Several Codes 

Originally we planned tQ. compare the best - 
version of • the -SA-AB code (i .e. , the code result- 
ing after testing alternative ^staft and .pivot 
rules) with other, cod.es which are based on other 
algorithms. Unfortunately, we*"did not have time 
to test alternative start and pivot procedures. 
Thus, the following code- comparison is a worst 
case comparison. That is, -it is 'comparing a 
computationally unimproved version of the SA-AB 
code with the best version of other codes. 

The codes wliich^we obtained for comparison 
include. Bennington [6], BSRL, General Motors, 
SHARE, and SUPERK [3] . All of these codes are' ^" 
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variants of the out-of-kilter. method except for 
Bennington 16].. In addition, the special purpose 
primal simplex codes, ARC- II .[5]/ ,PNET-I [11] , ^and 
SUPERT-2 [2] were available. for testing. Further* ' 
the special* purpose dual . simplex DNET [11] Ws 
available for testing. : j 

The BSRL code was dej^eloped by T. Bray and 
C. Witzgall at the Boeing Scientific Resear^^h, 
Laboratories, and the General Motors (GM) code was 
developed by Rand Corp. and is distributed /by the 
SHARE user group. ; ' . , 

All of chese codes are in-^core codes/ i.e. ,^ 
the progrsun and all ' of the problem data slmul- * 
taneously feiside in fast-access memory. /They are 
all coided in FORTRAN and none, of them (including 
the. special purpose primiai and dual simplex codes)' 
have been tuned (optimized) for a- particular com- 
piler. _ It is important' tp' note that all the cpdes 
except for SUPERr-2 and SA-AB codes are designed 
to solve capacitated transshipment problems and ' 
are not., specif ically designed ; to exploit- the 
special structure of- semi-assigment problems^ 
Further, SUPERT-2 is designed to solve any uncapaci- 
tated transportation problem. All o% the problems 
were*=solyed on the CDC 6600 at . the University of 
' Texas Computation Center using the RUN compiler. 
The computer .jobs were executed during periods 
when the machine load was approximately the same, 
and all solution times are exclusive of input and 
output; i.e., the total time spent solving the ^ 
problem was recorded, by caljling a Real time Clock 
upon starting to solve Che problem and . again when 
the solution was obtained- . . ^t-^: 

- ■ In addition CO these cod^," the articXe^by^* 
Hatch [15] compares a primal-dual code PD-AAL^ 
developed by Decision Systems Associates (DSA) 
against PNET-I on five assignment problems gen-, 
erated by NETGEN *[17]- Since the DSA code;^i.s 
proprietary, we. coxild not obtain a copy of it for 
comparison. -However, with their pub^lished times 
on a CDC-6600; we felt that some comparison could 
be made. if we Tan the same problems' on a CDC-6600. 
Thus* in order to compare our results with the ' 
results of [15], we solved the same five assign- 
ment problems. These results are contained in ' 
Table I. When comparing these results, it is » 
''important to ^te that we are^not sure that the 
code PD-AAL is an all FORTRAN code and it is our 
understanding tha^ the code is fully optimized ' 
to exploit the special hardware features' of - the ■ 
CDC-66P0. This type of specialization could. ; - 
easily increase the performance of all • the other 
codes by a^factor of 2. or 3. ... 

■•A notev^rthy feature, of the qpmputational 
results is that SA-AB, FD-AAL, SUPERT-2, and 
ARC-II are . in this order superior: to the other 
codes* Based on\the sum of the solution times, 
' SA-AB. is* roughly fifteen percent faster than its 
closest > competitor and is-, roughly fifty percent 
faster than its next closest competit;or. Fur- 
thei^,.. the computational results indicate that the 
AB algorithm reduces the number of pivots by 25Z 
over* the simplex algorithm.- Additionally, the 
results indicate that the, AB algorithm not only, 
reduces the number of degexieratfe pivots but also 
reduces the/ number . of nond^generate *pivots per- 
formed on the denser problems. Moreover, the 
reduction in the nondegenetate pivots (that is, 
the numbex/^of extreme points visited) versus the 
simplex algorithm increases as the number, of arcs 
increases. . • , * 



These results lead us to believe that the AB f 
algorithm may be the fastest' algorithm for soiling 
assignment problems;.- * ; ^. " • ( 

Further, these resets indicate that thi^ - « 
f±rst implementation of *the- AB algorithm is 1 
1 1 / 2 times as fast as the fasj&est un capacitated 
primal simplex transportation code SUPERT-2. 
Historically^ it has always been possible to 
improve the solution speed of the first implementa- 
tion of an; algorithm by a faotor of .2 or 3. . Thus, 
coupling this with the fact that the start and 
pivot rules of SA-AB have not been computationally 
investigated, it appears that the AB- algorithm . 
is probably twice as fast as other algorithms for 
solving assignment problems^ .This result is 
extremely important since it completely con tra- 
.^icts the older folklore that . primal -dual and 
out-of-kilter algorithms are the fastest and the- 
more recent folklore that special purpose primal 
simplex based codes are the fastest. 

Looking at the out-of-kilter and dual, simplex 
codes' solution times, it is interesting to note 
that these solution times are much slower than the 
other codes; further, their times- -are much more 
dependent on the number of *ai;cs (holding all 
other parameters of the problem 'constant) than * 
the other^'codes. Another important result which . 
can be gleaned from Table. I is that the dual 
simplex method is nof competitive with any of the 
other algorithms. 

■After- comparing all of the codes on the • - 
assignment problems., we choose from available codes 
the,::i^ree fastest ones (note that the proprietary 
PD-AAL code was not available) to compare on semi- 
assignment- problems'- In distinction to the assign- 
ment test problems, the specification of the semi- , 
assignment- test problems vary greatly in both the 
number of nodes, and arcs... -As shown in Table II,' 
the eleven test problems vary in size' from 50 . 
origins and^SOO destinations to 400 origins and 
4000 destinations. The number of arcs varies . " 
from 2000 arcs"*.to 20,000 arcs. The' cost range 
of the test'problems'is 1-1000. ^ 

The solution times in Table II again indicate 
that the AB algorithm is substantially superior • 
to the simplex algorithm. The SA-AB times strictly 
- dominate, the times of the other codes. . Comparing ' 
the sum of the total solution times, the SA-AB 
code is. 2.55 times faster ■ than one .of the fastest 
simplex transportation, codes, SUPERT-2. This is 
*a rather startling result since some people have 
indicated that they believe that the speed of the 
^simplex phased codes ^aire-approaching the computa-' 
^clonal limits^, of these problems 

• Another noteworthy * feature, of the . com^futa- 
tional results is that, the . SA-AB. code is decidely 
supexiOr on the largest. test problems. . Consider 
the last two test problems-which, each have ^400-;. - 
origins aryi ."4000 destinations:. On these problems',, 
the. SA-AB is a full three times faster than the - 
SDPERT-2 code.... * 

Based on the results, of this testing, .the • 
fact that the SA-AB code is the first implementa-. 
tion of the AB algorithm, and the fact that the**^ 
SA-AB code has not been computationally refined 
^ (i.e. , alternative start and pivot rules were 
■ not examined) , we believe .that the AB algorithm . . ' 
'is currently the fastest algorithm :f or solving . - ' 
5. both assignment and semi-assignment problems. ' 

This result is extremely. encdiiVaging since we have - 
recently. .extended the concepts ' of this algoi^ithm . " . 



.TABLE J. ... ■ 

. TOTAL; SpLin:i.ON,TpiES ON 200 -X- 200 ASSIGNMENT: PROBLEMS .. 
(IN SECbNDS)' ON A* CDC;66O0 WITH A COST RANGE OF 1-100 



No^f ARCS 


1500 . 


2250 


. .3000- .■ 


>■ ^375'0. ^ . 


4500 


t . 

1 Sum of Times . 


ARC ii 


1.45. 


vi.95' 


2.47 ' . 


2.67, 


3.13 

V 


11 '.67 


■ BENN 


17-44 


20.31 


. 24. 92* 
* 


27.40 




BSRL 


' 30-. 39 


. 22.08 


20.02 


•'23-11 ■ 


21.08 


116.68 


DNET : \ 


- 19-87 


.26.58 


27.98 


\ 30;i5 


■ 31,57 , 


' .136.15 


' -GM- - 


35.67 


28.4.3 


31.39 


18.62 ^ 


23.48 . 


137.59 


' r-PD-AAL - 


1.63^ 


1.14 


^1.89 , 


: i-.29 .:. 


. 1.30 . . 


.1.15 - : .. 


PNET-I 


2.31 


3.71 ' ■ 


3.47 


3.44 


4J79 


17.72 


SA-AB 


.97 


1.12 ■ 


1.48 


1.61 


1.68 


6,86 


• •' SHARE 


19.93 


21-17 


25.81 


24.95 ■ 


27.05 


118.91 


. ' SUPERK 


6.44 


6.47 


7.25 ■ 


6.95 


. 7.56- ' 


^- 34 .67 • 


.SUPERT-2 r . 


■ 1.26 


1.57 ■ ■ 


1.98 


2.17 . 


2.53. 


9.51 . 



TABLE II 

. TOTAL -SOLUTION TIMES ^ON SEMI-ASSIGNMENT PROBLEMS 
(IN SECONDS) ON A-CDC 66^00 WITH A COST RANGE .OF I-IO6O 





. Nd. o£ Nodies nr X n 


No. of Arcs 


•ARC-II 


SA-AB- 


SUPERT-2 




50. X 500 


2,000' 


2.21- 


1.14'- ■ 


2;85.. ' 




50 X 500 ■ ^ 


• 5,000 * - . 


- 3,53 . 


2.29 


.3. 51 




..; ' 50 x"500 ' 


io,ooo ' 


6.56 


4.00 


5.53 




50 X lOOO- 


■ 4,000^ • • . 


. • 4l27 . 


: 2.75 


■ 6.-15 _ . 




: . 50 X 1000 


, 10,000 .. 


/ 9.34 


■■ ^ .5.64. 


■ 11.64.' 




50 x lOOO .. 


20,000; 


. - DNR ■• 


■ y8.17' 


11.02 




100 X 1000 


.4;'000 


5i'59 . 


3.20 ' ■ 


6.22 




100x1000 


10,000 


" 10.25 


5.62^ 


10.99 




100 ^ 1000 ' 


16,000 


15.40 . ' 


8:16 


, ■14.27 




400 X 4000 


• 10,000. . 


DNR.. ' 


21.47 ' 


61.13 




- 400 X 4000 


■ 16,000' 




27.81 


, 91.27 




. SUM OF TOTAL. TIMES 






90.25 


230.58 



. DNR — ^Did not ' run as a result of memory/limitations-^ • ; , 
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■ ' TABLE III 

CODE spTecificaxions: 





." Developer . 


, Name 




■ Number of Arrays 


1, 


Barr 




Primal Simplex Transportation 


' 5 (M + N) + 2A 


2. 


Barr", Glover, 


ARC-II 


Primal* Simplex Network 


7 (M + N) + 2A 




Kllngman 






3. 


Barr, Glover, 


SA-AB 


AB Algorithm - • 


•' 4M +'2N + 2A . 




. Kllngman . . ' 


'"'SUPERK 






4. 


Barr, Glover, 


. put-of-ki5Lter ■ 


4 (M + N) + 9A 




Kllngman 






5. 


♦Bennington'- - 


• BENN - - 


Non-simplex . 


6 (M + N) +11A 


6. 


tray and Wltzgall 


- BSRL . : ■ 


•Out-of-kilter' ■ 


6>(M + N) + 8A 


7.. 


Clasen ' . 


SHARE 


. Out-of-kflter 


.6 (M-H N) + 7A 


8. 


Decision System 


' PD-AAL' 


Primal-Dual ■/...',"/ 


^ot available . ^ 




Associates 






. 9. 


Glover, Kamey, 
Kllngman 


DNET 


Dual Simplex network 


■ 7 (M + N) + 2A 


10. 


Glover, Kamey, * 


PNET-I 


Primal Simplex network 


6 (M + N) + 2A 




Kllngman, S tut z 






11* ; 


General Motors 


GM 


Out-of-kllter • 


3' (M + N) + 5A 



M — Origin Length 

N— Destination Length 

A — Arc. Length 



■to arbitrary capacitated transportation, trans- 
shipment , /and generalized transshipment problems- 

6-2 Memory Requirements of the ^odes 

Table II L indicates the number of origin, 
destination, and arc length arrays required in 
each of the codes testing for solving assignment 
and semi-assignment problems except for- the 
PIKAAL code. The istorage requirements of this 
.code were not available- It should -be noted that' 
memory requirements of all of the codes tested 

^ were -quite close (within -1500 words) excluding 
the array req^iirements - Thus, the important.- ■ 

- fac^:pr.Mn comparing the codes is the number of 
origin, destination, and arc length arrays-' . 

■ Looking at Table III and keeping in mind . 
th^t any meaningful problem has to have more 

, arcs than nodes, it^is clear that the AB'; primal . 
simplex, and dual simplex codes 'have- a distinct 
advantage (in terms of memory 'Requirements) over . ^ 
all of the other codes- Further, this advantage 
greatly increases as the number of arcs increase - 
For example, consider a problem" which has 10 
times as many arcs' as no'des- ARC-II , - DNET, PNET-I, 
SUPERT-2, or SA-AB require only about oner-half ' 
the' memory. that the- best (in. terms of memory 
requirements) of the- other--codes. This enables ■ 
the AB and simplex based codes to solve much 
larger problems than other codes-. Further^ it 
is important to note that the -^AB based code, SA-AB,' 
requires the. least amoxmt of memory- Thus, it 
appears that the AB algorithm is superior to other 
algorithms 'both in terms, of solution speed and ■ 

♦computer storage requirements- ■ . \ 
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Abstract 




' The vehicle routing problem has be^n regarded 
' ^ thorny mathema'tical programming problem for 

:. quite . some time. Hbweyer, only, recently . has. this 
. ^practical problem received widespread attention' in 
the literature and research efforts in this area 
continue- Largely this stems from the emergence of 
Algorithmic efficiency as a major concern in pper- 

' ations Research- "in addition, the new respecta- , 

.■ bility of /heuristic approaches in response to .the . 
■ la^rge number ef intractable NP- complete problems 
has * attracted many^ researchers- ' In this paper, we^ 
survey some of ' the^^ecent developments in vehicLe 
routing.' We focus oii broad issues such as, for 
example/ computational experiments with various" 
algorithms... For vehicle routing problems, ' tMe 

.. number feasible, solutions is 'a key factor in 
algorithmic performance- We demonstrate an ap- 

.• proach for .ceQculating thi^ number, ^'inally, var- 
iations pealing with . arc -routing and extensions " 
concerned with scheduling are introduced- i 

. * -~ ' - Introduction - ?. - . . ■ 

The Veibide Routing problem (trRP).^^seems first to 
! ; have been jqentiohed by/Garvin et^^. [-14] in 
:195X.anci by Dantzig and Ramser [llj in '1959- 
.Hpwever, only recently has this problem received 
. - .widespread attention in ^he literature and ' 
' research efforts in this' area continue. Largely 
thLs stems f rem the emergence of algorithmic - ' 

implementation as a major concern in Operations 
Rese«ch. In addition',, the new respectability 
..'of heuristic" approaches in, response to the large 
number of intractable NP-complete problems suc^ 
.as the knapsack problem* and the Traveling Sales- 
mem Problem, has attracted many rese'krchers. In 
this pajper, we sxirvey some of the historical 
development^ in the vehicle routing literature. 
The intentioir is not to be encyclopedic; -the sur- 
veys by Bodin [5} Christofides [7J ^ Golden [1.7], 
and Turner et al. ^38] discuss many details 
which we will not cover here. Rather, we hope 
to focus on broad issues and recent developments 
and provide an- overview. of vehicle routing. In 
addition, we count feasible solutions to the VRP 
and touch upon several new directions in vehicle 
routing research - 

Vehicle Routing Problems, sometimes referred to as 
truck-dispatching problems, are almost always en- 
countered by complex organizations , ^nd reliable 



procedures dealing with them are' needed. - Re- 
^ cently, higher vehicle costs dae to increased oil 
prices and rising truck dr^Lvers' salaries have mo- 
tivated, management to study these issues more care 

•fully... . ■ :• . ^ . ; ■. ^ 

There may .be several hundred demand points *in and, 
around a city^ The Vehicle Routing Problem is to' 
obtain a set of delivery routes from a cer^ral- de-. 
pot. to Tzhe various demand points ^ch of which has 
known Requirements , which minimizes th^ total 
dis.tance covered by-'the entire .fleet.** Vehicles .,- 
ivave caps^ities and maximum route time cons^;raints , 
.In addition, Jthe fleet of vehicles may be hetero- 
geneous with respect to these characteristic^. . It 
should be noted that there are a number of goals 
■ involved which include minimizing' the .n^Scnber of 
Vehicles required in the flee^, minimi^ig travel 
time or travel distance by .velJicles, AncJ^^enerally, 
providi-ng efficient service to the customers as 
quickly and. -cost-effectively^ as possible'.' The ^ 
^s'pecific objectJ:v^ functi^on we -consider explicitly. 

' vCminimize travel distance) is laoth reasonable and. ■ 
tractable. All vehicles depart from the cenEtal 

.' depot, make a toXtr of a- suijset of the demand nodes, 
and return to. the central »depot. ' 

Examples of Vehicle Routi^ig Problems- include muni- 
cipal waste collection studied by Beltrami &nd " 
Bodin I3J , ' fuel oil- delivery^ studied by Gartf^jfcet 
al. [14] , newspaper distribution stuc3ied by ^^ten^ 
Magnanti, and .Nguyen "[19] ,'and routing of schS^ 
buses studied by- Newton and Thomas '[26] .. It should 
be clear that ^1 of these vehicle routing problems 
are more- or less the same^ dperationaJ?ly the 
examples may seem different^ but theoretically 
they can .be thought of as e<:jnivalent . ' 

- .■ . • « 

Proposed techniques for solving problems of this 
sort have fallen into two classes — those which 
solve the problem optimally by branch and bound - 
techniques (Christofides and Eilon. [8], Eilon et ^ 
al. [12], and pierce [30]), and thosev^which solve 
the problem heuristically ' (such ais Christofides 
and. Eilon [9] , Clarjje. and Wright [10] , Dantzig and 
Ramser [11], Gafekell [15] , 'ciilett and. Miller , [16] , 
Golden, Magnam/ci, and Mguyen [19]Y Holmes and 
Parker [20], Russell [34], Tillman and Cochran' - - 
[37], Tyagi [p9J , cmd Yellow*" [41]). *in a loose 
sense, heuristic alcforithms represent* sets of rules 
which produce good soli^ions to given combinator- 
ial-programming problems^ but hot necessarily the 
best possible (optimal) • solutions . Since 
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aiXgorithns which are guaranteed to achieve optimal- 
ity ar«" viable only for very small problems, most 
• authors prefer to concentrate on the study of . 
hear is tie algorithms". ' , ' ' . . 

. S"^ ' <.> • • : • • ■ ■ ■ 

• VRP Tormulat^jon 




Garvin et al. [14], who- treated an application in 
the oil industry, introduced admixed integer- 
; programming heterogeneous fleet problem formula- 
tion of Ithe VRP.. This formulation has on the 
order of n^ constraints where n is the number of 
nodes.". 

Alternatively^ when the fleet is- homogeneous , the 
VKP niay ise expressed as .the following large-scale 
iine£ur-integeJG program. * , - 

■ ■ .-. • ' ■ ' 



"^nimize 



Subject to 



1 6 X = 1. 
.1=1 ^ . ■ 



1, . . . rn 



lilcely., however, that all NP-complete problems are 
intractable and any optimal algorithm must be of at 
least -exponential time 'complexity* _ 

' We present below a complete linear-integer program- 
ming. VPP formulation ^with structure moreU:losely 
related to the fundamental , JSP formulation- This 
formulation was introduced by Golden, Mag6anti, and^ 
Nguyen [193 - ■ ■ " ^ .... 

We point out that in the otherwise excellent survey. . 
by Turner et al* [38] , an incorrect integer program- 
ming' formulation is given. Unfortunately, th^ir 
formulation fails to prevent the formation of sub- 
tours. We will refer to the following formulation 
■ as ,the-Vl?P formulation: 
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where 



Jl if i^.^ 
< tour y 
\o btherwi 




customer xs on 
otherwise 



if tour j is chosen . 
otherwise • 



Cj • = the .length of tour j ■ 
n = -number of nodes 



m = ■■ ■ total number of .feasible 
' ■ tours •* 

V • .. " ■ " . - ■ • ■ 

This set partitioiSing formulation was first given 
by Balinski and iQuandt [2] in 1964. The ■ objective ^ 
function minimized total* distance traveled. Clearly 
such a formulation i-s most valuables for very small 
Wehicle Bouting Problems due to the large number o'f 
variables (feasible toxirs). . ^is formulation may 
be usef 111 as conceptual 'tool, whereas the formula- 
• tion given by Gotvin et al. i^s more explicit and 
might he better suited for integer programming, 
techniques, in worst case" 



m ■ 



■k=l ^ 



2^ - 1. 



k=l 



For example with n « 25, there may be more than 33 
million feasible- toiirs. Balinski and Quandt report 
success in solving TO^'s using a cutting plane algo- 
rithm with, problems for which n ^ 15 and m < 300: , 

The. set partitioning prciblem belongs to the NP-com- 
plete problM class along with the Vi?P. A problem 
is NP-coraplete ^f it can be shown, to be equivalent 
to the Traveling Salesman Problem (TSP) in the 
following sense. If one can be solved optimally • 
by i polynomial algorithm, then l>oth can be solved 
by polynomial algori»thms.* A polynomial algorithm 
is one in which running time is proper ti6pal to a 
polynomial function of the input. It seems 
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- (for 2 £i'?^ j ^ n for some ^eal 

numbers y^) • 

= 0 or 1 . (for all i,j,k), 

where - -n = number of nodes % - 

Nv'- number of , vehicle's ■ ^ 
. Pj^ = capacity of .vehicle k^. ^ ' 
,Tj^ = maximum time allov/ed for route of 
' vehicle ^k ' " 

^Qj^ = demand at node i (Q^ ^. 0) 

" t^ ss time required for vehicle k to 

deliver Or collect at node i (t^^sgf) 
' t^; 7= travel time for vehicle" k from 
^3 '^node' i' to node "j (t^^^ = <» ) 
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*d^j ■ distance from node i to node j 

(l if arc (i,j) is traversed by . 
jc!* "1 vehicle k 
i j . (o otherwise. - 

Node- 1 represents the central depot. Equation ' (1) 
states that total distance is to., be minimized. 
. Equations (2) and (3) ensur^, that each demand node 
is served by one vehicle and only one vehicle. 
PDute continuity is represented by equations- X4) , 

; i.e., if a vehicle enters a demand node, it must 
exit from that node. ■ Equations (5 J are' the vehi- 
cle capacity constraints; similarly, equations (6) 
are the total elapsed route time "constraints. For 
instance, a newspaper delivery truck may be re- 
stricted- from spending more than one hour on a 

■» tour in ;order that the maxilirum time interval* ftom 
press to^treet be- made as short as possible. 
Equations (7) and (8)' make certain that vehicle 
availability is not -exceeded. ^Equations (9) are 
the sub tour-breaking constraints - 

Since (1) and (4) imply (3) , and (4). and (7) imply 
(8), the equations (3) emd (8) are redundant, and 
can be excluded from the model; in any case, the 
linear- integer program is awesome in size. IVpi- 
cal pzroblems- involve hundreds of demand points*. 

■ Historical Survey 

Dantzig and Ramser [11] were the first researchers 
to obtain, a me'thod for solving the VHP approxi- 
mately- In 1964 Clarke and Wright [10] extended^ y 
the Dantzig and R£imser model to consider routing 
for a fleet, of vehicles, of vaurying capacities . 
The oew procedure botrowed the concept of node 
-aggregation from the earlier method and it seemed 
..; to yield- better results. Undoubtedly, the Clarice- 

■ Wright "savings" method is the most widely used . 
and cited vehicle routing algorithm. It in- 
volves first evalxaating all potential savings 
^ii ^li **" ^Ij " ^ij ^^o™ linking two nodes i 
and.'j, and then\3oining. those nodes with the, high- 
est feasible savings at each iteration. Initially, 
each node is served individually from the central 

■ depot. . This heuristic has been analyzed and modi- 
fied- extensively ; Gaskell [15] in 1967 experiment- 

-ed with another ■ savings function TT^j = '•■ ■dij - 
2dij with limited' success . In the above savings 
functions, dij denotes -the distance between nodes 
i and .j .• . . ^ • / 

Tyagi [39] in 1968 presented' a method*^hich groups 
danand points^ in the following very straightfor- 
ward fashion. Starting with node 2 (node 1 is the 
central depot) we find its nearest neighbor > say 
node k, subject to the. vehicle capacity restric-' 
tions (we assvme here that all -vehicle -capacities', 
are the same). We. next find the neaurest neighbor 
to node k, say node j, subject to the capacity ' 
restrictions and continue until adding a neairest 
neighbor would result in a tour exceeding the max- 
imum vehicle capacity. Rules, of thumb are speci- 
fiefd to minimize the frequency with which a group 
will consist' of onlyi-.one delivery point, .especial- 
ly, in the case where the delivery is^ small. or 
the distance, from the central depot to t&s* point 
is more thstn half the distance from the farthest . 
point- to the central' depot. ■ Having groupfed the * 
delivery points -into. -m tours, the- vehicle dis- • 
patching puiTOblem reduces to m ;rSP'sV one for each • 



. ■ tour... . Computational atspects of this" algorithm are 
discussed in Golden, Magnanti, and Nguyen [19].^ *^ 

In 1969,^ Christofides andEilon [8] and Pierce [30] 
presented- algorithms for finding optimal solutions 
to small VRP's. • Christof ides and Eilon ptesent a 
branch and bound strategy similar to the well-:- 
•kupwn Little etal. [24] approach for TSP's. . 
Pierce disciisses direct-se£u:ch, combinatorial pro- 
■gramming algorithms for a host of truck- dispatch- 
ing problems- Delivery deadlines, earliest deliv- 
ery time' constraints, carrier capacity constraints, 
optional deliver i^, and generalized -objective 
functions are developed in detail in his^ ambitious 
paper: ... 

Yellow [41], indealing with Euclidean networks, 
eliminates' the need to initially compute the half- 
matrix of savings, values by ■ applying a. simple geo- . 
metrical se£u:ch ' technique based on the polar coor- 
dinates of thedemcuid points- In this 1970 paper, 
^ he modifies the . sarvings method to produce a sequen- 
tial Cl£u:ke-Wright method where we proceed as in 
Tyagi 's algorithm except that maxlmiim savings 
rather than minimum distance is our criterion. 
That is, if we have j.ust linked node i to the sub- 
tour, we next find the largest feasiblie savings 
between node i and a nearby node j, not yet in' 
the subtqur . . , \ 

The excellent book Distribution Management ,^ pub- 
lished in ,1971, studies TSP*s and VRP's in great 
■ detail [12] . 'Eilon et al. include an "r-optimal" 
. procedure for the VRP which borrows ^uch from , 
* Lin's TSP approach [22]- Their procedure begins 
with a feasible solution and tests perturbations 
of , r arcs at a time to obtain. r-optimality. ' For 
example, if r = 2 they examine each pair of arcs 
to see if it can be replaced by another pair such 
that .feasibility is preserved and total distance 
is* decreased. Lin's approach has been expended by • 
Lin and Kernighan [23] in 1973- It should, be : - . ; 
noted that many of the book's computational; argu- 
ments are already out of date- ' ' ' 

Krolak et al- [21] in. 1972 suggested a^- man-machine ^ 
interactive approach which has been successful on* 
some larger problems but seems to require aii ex- 
cessive amount of man-machine time- In 1974, 
GilLett and 'Miller [16] proposed a "sweep" algo- 

. ■ rithm for Euclidean networks which 'ranks and" links 
demand poinbp .by their polar coordinate angle* 
Newton and Thomas have recently investigated the 
VRP in connection with school bus 'routing, dealing, 
with a mialti-school system [26]- Their approach 
for each school hsts been to obtain a neeur-minimum 
single trip solution to the relaxejd TSP, and . then . 

■ peirtitioji^he zTesialting tour' into 5hbtours 'fetis-' 
fying the VRP coi^straihts [25}'.'. ' ^ ' 

In Beltrami^ and Bodin's very comprehensive 1974 
^ pSper .[3] , a variety of problems .concerning' muni- * 
Qipal -waste collection are explored.* For example, . 
they consider the potential benefit of ■ allowing 
. more than one vehicle-^;to visit a .site on the same . / 
day, each of which .services part of the demand at 
the site- In addition, some* pathological features - 
of the Clarke-Wright ^heuris tics ^2u:e pointed o'ut- 

rin 1975, the emphatsis in vehicle roifting was on • . 
^.the development of codes to solve ia!?ge-S:ale . m 



VKP's. Orloff , who had previously formulated the 
General Routing Problem (GRP) , extended his model 
to handle a fleet of vehicles. The GRP concerns 
.finding a minimum cost cycle which^ traverses every 
arc and every node in subsets R and Q of the arcs 
and nodes respectively. We will discuss node 
routing and arc routing later in this paper. For , 
now-" we remeurk that the GRP formalizes this d*ich6t- 
omy and, at the same time, defines a continuum 
between these two extremes. Node routing ^ 
problems tend to- be much more difficult to solve 
than arc routing problems. Recently, Orloff and 
Caprera [29] have exploited this observation in 
'a heuristic algorithm Cof the Lin-Kernighan 
variety) for solving large GRP's. The strategy 
is to convert required nodes to required arcs *; 
Whenever possible. In other words / requiring . 
certain major roads in £kj||^ansportation network 
to' be traversed in a tour limits the degrees of 
freedom in an algorithm and reduces running times. 
Larg^ VRP's may be exploited similarly. 

In I974", Russell f333 presented a modified 
version of the Gillett and Miller algorithm- 
More recently, Russell [34] has presented a 
VHP h^^istic algorithm for the special case 
whe^re tfce number of tours is pre-specified. Also . 
various sequencing and due-date constraints are 
considered. The algorithm is an ext^ -i^ion of 
the 'Lin-Kernighan heuristic [23] . nuk. ' • 

solved a 159 node problem in unJcr 9 minute- 
on an IBM 370/168. Robbins.e^ al. [32], also 
in 1975, have assembled a tour construction-t: ' 
improvement code which, look? q*-ite pr^-. : ^ ing* 
Initially r a tour is construci.^ us^ng rhc 

, Clarke- wAght approach. Next, zr.i^ tour is 
improved, by Lin's 2-6pt procedure '22] . In five 
of the seven cases cited, -zjie improvement is less 
than 1%? the laurgest imprc-ement is 3.2^. A 

.150 node VRP, was solved in .ai^'ut 37 sec-onds.' 
This techniques -has been app^^^e by Vu and Turner ■ 
[401 to a'rural refuse collect ^-jn problem. Th<^ 
code presented by Golden, Magnar ti / and Nguyen 
[19]' performs from one to two ordors.of magnitude 

. faster than these recently developer; codes 

. although it may hot be as accurate . 

In their, paper, the authors emphasize data 
structures and list processing and present a new 
implementation of the Clarke-Wright algorithm 
. which is motivated by (i) op timality 'considera- 
tions, rii) storage considerations, and (iii) 
sortir.c considerations and program running time. 
The Clarke-Wright algorithm is modified in the 
following three ways-. - • . 

. (!)• by using a route shape parameter y to . * ; 
define a modified savings . . ; ;;* 

s^j ■= d3_i - " "^^ij and finding 

. ■ the. best rout structure obtained as 

■" -.the parameter is varied^ • js^, ■ 

■."^20 by considering savings o^^ly between 

nddes that are "close" to each other;. . 
; (3)tby storing savings S. . in a heap . , 

J structure' to reSuce comparison- 
. ^ . operations aj^d ease access. - 
.•Shis modiified algorithm ha^ been applied to a 
Newspaper.' distribution problem containing ^ 
nearly 600 .drop points. The problem was^'solVed 
in less thatr 20^ seconds, of ^ execution- time on an 
IBM 370/168. • ■ ' - ' - , 



Holmes and -Parker [50] , in 1976, have, suggested ' 
another improvemertt to the ClarJce-Wright algo- 
rithm. The modification entails a progression from 
one feasible solution to a next*, until ws find a 
local minimum with the following propert^v If we 
. -prohibit any one edge in the current .solution from 
appearing in the next solution (by setting its . 
•savings value to zero), and reapply the Clarke - 
Wright algorithm, we cannot improve upon the total- 
distance. In terms of accuracy^ this concept of 
local minimum seems to yield nice results. Com- 
putationally, however, this approach is relatively 
slow. 

Vehicle routijig algorithms have recently "come of 
age" in the sense that they are now capable of 
solving some large-scale real-world problems. As 
indicated in this survey, the development has been 
a slow process. We can categorize the chronologi- 
'cal stages of growth as follows: 
1 ) early formulations ; ' 
^ 2) early algorithms for hand computations? 

3) exact algorithms for small problems; 

4) - more efficient, computerized algorithms with 

an emphasis on implementation for moderate 
sizre problems-; , 
- 5) second generation codes, much faster— for 
large problems ; . ^ ^ . 

6) real*wor id applications - 
There are, of course, many important variants of 
the VRP, some of wl^ch we will mention, for which 
no satisfactory solution techniques yet exist. 

1 , ■ ■ ■ 

We remark "in closing this section that many of the 
papers mentioned are no more than .variations and 
combinations of the ."savings" method, the "sweep" 
method, the "near est- neighbor" method, and the 
"r-opt" method. ^ - 

■ ■ ■ ■ ■ .J . 

- . ■ Counting ;and the VR]P ^ . ^y* 

For hard integer programming problems such as the 
VRP where enumerative techniques are^ the fijRiary 
tool for- solving e.v^ the smallest problei& 
exactly, the number of feasible solutions is a. key 
factor iri algorithmic performance. .Generally it is 
impossible to explicitly enumerate and evaluate all 
, the feasible -solutions for all but the smallest 
problems. In"* this section, we discuss partitions 
and counting feasible solutions to the VRP. 

In order to get an idea of the number of feasible 
solutions to the' VRP, consider' a situation where p. 
: vehicles are available to serve n demancj^oints. 
Each vehicle can make no more than k scops due to « 
capacity constraints. The number of feasible 
solutions gives an indication of the complexity, of 
"these difficult combinatorial programming problems. 
We demonstrate an approach for calculating this 
number based on a simple recursion formula. 

As background,- we briefly introduce some notions 
about partition thjeoryl ;The theory of partitions 
isian area .of numbpr theory which deals with the 
reiaresentatloi^ p4 integers as sums o5 other inte- 

Definition Iz A partition of a non-negatxve inte- 
ger n is a representation of . n as "a siim' of positive 
^integers, called' either sunanands ■ or parts of the 
partition. The order of the summaiids is . irrelevant. 



The partitions of 5 are 5, 4 + 1, 3 + 2/3 + 1 + 1, 
2 + 2 + 1, 2 + 1 > 1 + 1, and 1 +1+1+1+1. 
Thus, there are seven partitions of 5. . We remark 
that 0 has one partition, the empty partition, and 
that the enipty partition has no parts* An excei- 
lent source on. elementary partition theory is 
Andrews [13 • V 

Theorem 1^ Let Pj^(m,n) denote the number of par- 
titions of n into exactly m parts, each of which 
does not exceed )c. Then 

P^im.n) ^^^^j^(m,n)+Py.iT[^l,n-k) for k^l,l<m<n 

^1 * for k^l,m«»n=0 

, vP otherwise. 

Proof ; The pzirtitions of n into m parts with no 
part exceeding k. can be divided according to 
whether or not k is a summamd.^ If it is not, 
then there are P]Q_i(m,n) partitions. If it is a 
.summand, then the remaining sum is n-k, and we * 
divide into m-1 parts not exceeding k, so 
Pj^(m-l,n-k) describes the appropriate number of 
such partitions.. These events are obviously ' 
mutually exclusive emd collectively exhaustive, 
and the recursion is obtained. Boundcury condx- 
txons<^are as indicated. ' . 

I)efinition _2: Let <}>3^(m,n) be the number of fea- 
, sible' Solutions to the VKP with m vehicles, n 
demand points, and a delivery -capacity of k demand \ 
E>oints for each vehicle. We assumer^ vehicles are ' 

indistinguishable^ • :. 

■ '. * > " ■ ■ > 

Definition 3: Let Sjc'(m>n) be the set -of all sets-' 
s « {p^ , -^PmJ such that P^ + P2 ft ^ ■ 

and-O <_ p . jC fc.- ' , . 

c" .m ' ■ - " 
Theorem '2; . <>3^(m,n) ^ tS^ T 



when n » 10, there are 3.6 x 10^ tours; when n = 50 
■^3.04 X 10&4 tours; When n ="100, 9.3 x 10^57 tours. 

We can see how comb in atori ally explosive TSP's can' 
be. Prom Jheorem. 2 we learn that^fqr a. problem - 
with n demand nodes the TOP has many niore feasible 
solutions. Suppose we have a situation with 6 de- 
mand nodes, 4 vehicles, and a- capacity' of 4 units- • 
There are a total of 6* = 720 TSP tours.. The 
eleven partitions of 6. are shown below! - 



6 

5 + 
4 + 



+ 1 



+ 1 
+ 1 
+ 2 
+ 1 
+ 1 
+ '1 



+ 1 



-Since <J>4(4,6) = 6! 



+ 



P^a,6) and 



n 



Proof 1 ,^(m,n).- ., . p 1'. .p OPiiPil - - -P„i 

' r seS^(m,n) ^1^2 :^m . . 

since wte must first distribute 'the n demand points 
am^ng the m,. vehicles (we need not use all the 
vehicles) and then we mxist order the points for 
each vehicle. The value of p^ is* the number of 
demand points assigned to vehicle i. So, 



• <>j^(m,n} 



seSk(m,n) ^pTTppTTT^jT^ 



{the 'jiumber qf partitions of , 
the Integer , n into *at**most m 
parts ^at have no part ex-- 
ceeding k} 



- n!. Z: P^(Ji,n) . ' ■ 

. . . y : jz^i ^ 

The Traveling Salesman Problem is a special case- 
of the VRP with m = 1 and k » n. <J)jj(l,n) [is 
given by <t>ni^,n) - niP^d^n) - nl For exaijple. 



2 'P4(t,6) =7 the 
- - ii.-l £=1 

number of feasible solutions to the VRP is 71 = 5040, 
In general, we can determine ,c()j^ (m,n) easily after 
first computing Pj^U,n) for Z ='l,2,...,m via 
Theorem 1. ■ - 

The Capacitated Chinese Postmsm Problem 

The problem.of.ifinding. an optimal route for a single 
vehicle over a network is ^ common ^id,.^ as we* have 
seen, very difficult combinatorial problem.' Rout- 
,cing problems" can be classified as- node routing 
problems, arc routing problems, or genera;L routing 
problems. 'Bbdin 15] '^provides a coriveni en t taxonomy, 
for these problems. The problem of visiting all 

■ nodes in a network in the minimal amount of time;^ ^ 

■ (node routing) is the classical TSP. -The problem ^ 
of covering all arcs of a network while nnLnimizing . 
the total distance traveled (arc routing) is the ' 
Chinese Postman Problem* (CPP) . The General Rout- 
ing Problem (GRP)' on network G = G(N?A) (N is the : 
set of^lj^nodes, A the set of all arcs) is a gen- ' 
eralization which includes the TSP and the CPP as 
special cases. Here we seek the minimum cost cycle 
which visits every node in subset Q £ N and cavers 

■ every arc in subset R c A. . This paper ^s primarily 
concerned with node routing problems. Orloff intro^ 
duces the GRP and later extends it to^ handle more 
realistic problem situations [27] , ^ [28]/ [29]. 

Applications of 'arc routing "problems include rout- ^ 
ing of street sweepers, snow, plows^ household re- 
fuse collection vehicles, postmen,' the spraying of . 
roads with salt-grit to prevent ice formation, the 
inspection of electric power lines, gas, or oil 
pipelines for faults, etc. Since in many vehicle 
routing situations customers are so numerous that 
identifying them individually becomes* cumbersome, 
we caiy -consider the CPP. to be .the .continuous 
counterpart to the ^iscrete TSP. Here -an 2irc^ 
replaces a number of customers - 

In the CPP we assume that all arcs are unddLrected. 
Strieker [36] and Christof ides 16] consider an 
extension which we will call "the , Capacitated 
Chinese Postman Problenh" (CO^P) which, reflects 
^^^l-l^fe situations more dirrectlyi We eire ' given • 
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Minimize 



1 
i-1 



Subject, to 



■ n . 



k-1 



for i*i,.^.,n 



(12) 



j-1 



arc demands Q " which must be satisfied by Minimize 

vehicles of capacity W. A niimber of cycles, must 
be formed which traverse every arc, satisfy de- ^ ■ 
mands-, and yield minimum distance. This is eui sure subject to t 

routing version of the VKP. Strieker and Christo- ■ 
fides have suggested heuristic procedures for ob- • , 
taining reasonable solutions to this problem. How- 
ever, an optimal solution appears to be^very dif- ■ 
ficult to obtain for all ^but the most trivial • . 

problems. We give an integer programming formula- 'where 
tion for'^e CCPP below which indicates the inher- 
ent complexity of this problem. / 

n " n NP / . • 

^ cijx?j . ; (11) 
■p-i . ■ - 



I' a. .X* > 1 for all i 



Xj'e{o,l} 



for all J 

^ the distance associated with tour j' 



4 



1 if feasible tour j is used' 
0 otherwise 



a. . 

ID 



i 



1 if arc i is covered in feasible 

tour j 
0 otherwise ► 



the number of: feasible tours . 



NP 



Z (rF + x^.) > 1 for all (i,j)eA 
p-l ^3' 



(13) 



NP 



Z + Jl'; ) - 1 for all (i,j)eA 

p-1 \ (14) 



x^ > for all (i,jjeA and 

^1 . -^'3 p » 1/:.., NP.. 



(15) 



Z- Jl?.q^. .< for p«l,- A.NP (16) ■ 
i«l j-1 "^^ . . ■ 



x^j > 0 and integer 



iPje{0,l} 



(17) 



(18) 



.whe'ro NP » the number of available postmen 



As with the VRP,, the number of feasible .tours can 
be enormous. The CCPP seems to belong to the same . 
problem class , as the TSP and the YRP. Any optimal 
aJ^orithm for its solution will most likely, be ex-' 
ponential. ' ' 

Christofides [6] presents a heuristic algorithm 
which appears to be efficient and effective.' Com- 
putational results are presented for graphs. with 
up to 50 nodes and 125 arcs. Running times are 
less than 30 seconds and the percent deviation from 
a' lower bound on the solution is less than nine in ' 
all cases-. Details of the algorithm are clarified: 
and an example is worked through in Christofides 

We stress the fact that, the CPP and CCPP are the 
arc routing coiinterparts to the TSP and VRP. A , 
primary conclusion is that the CCPP is, indeed', a 
practical problem which deserves much more research . 
attention^ ' ... 

■ Vehicle Scheduling 



xP »• the nimjber of times arc (i,j) is 
^3 traversed by postman p 

"5^ «A post^ian p services ^c- (i,j)^ 
Ij tp otherwise ^ 

, q^j* the demand on aurc (if j) 

W .« the vehicle capacity. 

The objective function (11) seeks to minimize dis- 
tance traveled. Equations (12) ensure route 
continuity. By (13) , every aurc is covered at 
least once. Equations (14) state that each ^eurc 
is serviced exactly one time. Equations (15) 
guarantee that arc (i,j) can be serviced by post- 
man p only if he covers arc (i,j). Every demand 
is satisfied, by equations (16). For the small 
example provided, by Christofides [6], with 12 nodes, 
22 arcsv and 5 postmen, the above formulation re- 
qxiires'329 constraints and 440 integer variabiles. ' 

Earlier in this, paper, we mentioned that Balinski 
and Quandt formulated the VRP as a set partition-., 
ing problem. Similarly, we can view the CCPP as- 
the following set covering problem: - 



We note that any , logistics system must be composed 
of both^ routing and scheduling components. Al- 
though the focus of this paper is on the^^outing 
aspect (the time dimension has so far been ignored), 
we now develop a framework for attacking the two 
problems simultaneously. The problems have been 
treated sequentially in Golden ^[18] in terms of the 
well-known cutting stock problem. , f 

An alternate, and as yet unexplored, means of con- 
necting the routing and scheduling components is 
through what Simpson [35] refers to as ^ schedule 
map. Whereas a route map considers a s^tialf geo- 
graphical network, the schedule map includes a time 
dimension as well. Simpson reports on models for 
ptablic tram sport ation systems, such as airline sys- 
tems, for which this distinction is valxiab]^. In 
the figiires below this dis'^fnction is illustrated. 




Figlire la: Route map' 
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Inrtthe schedule map, each node repreisents a. geo- 
graphical location and a specific -time. We "dis- . 
creti^e the* time axis by dividing the basic unit 
(perhaps a. workday) into a- number of smaller units 
or periods- We can imagine, in the schedule map " 
alSove, an eight- hour worJcday broken into hour, 
units. The time points correspond to 9i.30, 10r30, 

11:30, ,4:30. The distances on the route map 

stand for travel times in. terms of these hour. - 
units. For example, the travel timq^rom A to B 
is approximately one' hour, from B to C takes ap- 
proximately two hours, and so on. The number of 
time points for 'each location^ Cin this case, eight) 
is^chose^. such that the' travel times are appropri- 
• atei. Note that in.* the schedule map presented, we 
have split location A into two* locations A' and 
A" in order to make the network more readable. 

Once we have constructed^ a schedule map for the 
VRP we can proceed to route and schedule simul- 
taneously by applying a sequential Clarke-Wright 
algorithm. Savings may be defined in terms of 
travel times between nodes, or in terms of travel 
■distances. When a-'node i has just been included 
in a tour, we erase' alT other nodes which cor- 
respond to the same location but different time, 
/and then find another feasible node j (not yet 
in a tour) which yields the best savings s . . . 
By feasible, we mean that capacity constraints 
are not violated. Routes Rpgin from the origin^ 
at various time points in conformi1:y with the 
vehicle availability limitations. 

This model leads ixs'^o a Generalized -Vehicle 
Routing Problem CGVRP) . in the QTOP we ar^ 
interested in routing vehicles over the ^collec- 
tions Sx, Sj^ of nodes from a given origin?' Each 

node has a spgcj.fic demand. We require that at 
least one node in each collection be serviced ' 
and . we. "seek to minimize tot^ distance traveled. ■ 
-The schedule map. is one special case of -the GVRP. 
This general problem has 'applications- in many other 
routing, sitiiations where, -for instcince, there* are 
a number of control pffices or warehouses in each 
small area of . a much larger region which- must be 
serviced. ^ 

The^ difficult^jwith* a schedule " map "is evident. * ' 



Namely, the number of nodes and arcs to consider 
grows very rapidly once the time dimension is 
depicted. In any case,. the model itself is helpful 
for undjsr standing the complex relationship between 
the routing and scheduling components of a logis- 
tics 'system-- 

Conclusion 

, In this paper we have provided a concise overview 
*of vehicle routing. The enormous number' of fea^ 
sible solutions to these integer f)rogramining pro- ' 
blems becomes a factor in algorithmic performance.. 
We demonstrate how one might determine this number. 
Two new directions in vehicle routing research have 
been proposed: the first is an arc routing problem 
■ini-xmeLZely related to the .VRP — the Capacited 
Chinese Postmark, problem; the sec6nd is a framework 
. for envisioning, the interaction between- routing and 
scheduling via the schedule map—The Generalized • 
Vehicle Routing Problem. Thereare, of course, 
other important new research areas in vehicle rout- 
ing such as the evaluation of heuristic algrorithms 
which we have . not discussed here. 

In brief, the indication is. that some of th^> sug- 
gested procedures can be used as effective deci- 
sion-making .tools for large-scale Vehicle Routing 
Problems encountered in many practical situations 
in both the public and private seftprs. ^ 
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Abstract * • * 

Since the introduction of the additive algorithm 
. by Egon Balas in 1965 [1], numerous investigators 
have added reflrtenents and improveznents to the 
basic procedure to improve the convergence proper* 
ties- of the algorithm.' These embellishments have 
included procedures for changing the search origin, 
alterl^ the criteria for augmenting variables", ; 
etc.^,. and have included the use of composite con- 
straints for fathoming partial solutions early in ' 
the augmentation phase of the algorithm. In this 
paper, we will present computJational experience ' 
with many of these, refinements for the binary as 
veil as the bounded integer problem. 

Introduction 

A zero-one integer linear programming problem can 
be written in the form': ;■ * 

minimize: cx , ^ (1) 

- subject to: h -h Jix ^ 0 



where c and x are n- tuples, b ,is ap. m- tuple, and A 
is an m-x n 'matrix.' Form. (1) is termed the stand- 
ard form^ Any zero-one programming problem can- 
be written in form (1) by a suitable definition of 
; variab ies and manipula t ion of cons tralnts • , 

A bounded integer linear programming problem can 
be written in the form 

minimize : cx '* ' ,^ (2) 

subject to: ^ b + Ax 2: 0- 
"b ^ x^ d 
' integer 

' ' - c, i 0 

• .. • . ' ^ 
where d is an n-tuple, and the other variables 
are as defined in (1) above. Suitable trans- 
formations of variables and constraints can 
convert (if necessary) any bounded integer linear 
programming problem^ into form (2). 

procedures effective in solving (1) can be ^sed • 
^in solving (2) through a binary expsnsion'of the 
integer variables, Similarly, (1) is a special 
case of (2) in wiiich d^-I, j-l,^, There ' 

exists widespread interest in obtaining solutions * 



to (1) and (2) because of the wide variety of probr 
lems which can be formulated as zero*-one and . ^ 
. bounded integer programm;ing problems (see, for ' 
' example, [91, [211, [22], [26]). . . ' 

In this paper, we will evaluate several extensions 
to the basic Balasian algorithm for solving four 
classes of integer programming problems. These 
classes include capital budgeting, problems of the 
Lorie-Savage variety [21]; single-constraint allo- 
cation problems from Trauth and Woolisey [27], 
^ £lxed**charge or bounded integer prograioning prob- 
lems from Haldl [14], and "IBM test" problems 
also from Haldl and from TrautH and Woolsey, We . 
will not concern ourselves in this paper with / 
branch and bound procedures of the Land and Doig 
variety for solving the .Integer programming- prob- 
lem, as these methods represent fundamentally 
different approaches. Computational results repor- 
ted do, however, present the- opportunity for com- 
parisons between approaches (depth "first search/^ 
vs. best, first search}* - 

Many investigators who have suggested refinements • 
.to the Balasian enumeration algorithm have also 
made available FORTRAN computer programs for tes- 
ting their refinements [12], pLS]^ [18], [24], ; 
[29]*, An empirical investigation of each of these-- 
improvements -remains to be completed.^ In the i. .1^. 
next sect ion,, the computer programs and.,modif ica-^' 
tions examijjed are briefly described. Each^prb- c 
gr^ examined is basically an enumeration^ algorithm 
of the Balasian type with certain improvementis 
and modifications .to eliminate large portions of 
■ the search. Section III discusses the computa-'. 
tional experience observed in solving oategor ies 
of problems with the available techniques* Sec-- 
tion IV summarizes the results of .the investiga- 
tion and dis^cusses future improvements to enumera- 
tion algorithms which appear promising** - 

Computer Codes and Modifications Examined 

In this section, each of the computer codes 
(techniques) investigated is briefly described. 
Each of the codes implements an enumerative ^search \ 
procedure which is a 'basic Balasian algorithm 
with modifications designed to exclude portlons^^f 
the search (ia^licit enumeration). Each of the 
codes' is redimen s loned (x^pward) from the Ir ori- 
ginal versions^ to fit into 280 by tes of core. ^ . , 
" Each code is written entirely in '.the FORTRAl^ '. " 



language and la an "in-core" program. Dlstln- 
galshlng features or characteristics of each of 
the 'four codes cacamined arc summarized in Table 1. 



The strategy choices made by the user concern 
the frequency with which a surrogate constraint 
Is computed and the 'ifiaximum number of surrogate 
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^IF30C (Geoffrion and Nelson [12]) 

, The. code of Geoffrion and NelsoiX eiiq>loys a basic' 
Balaalffla algorithm and an imbedded L.P. to pro- 
. vide an initial partial solution and to con^jute 
a- strongest" surrogate constraint (as defined 
by Geoffrion. [9]). The 'major advantage of the 
particular surrogate constraint introduced by 
Geoffrion and incorporated into RIP3bc is that 
•the dual of - the required linear ' progrto coin- ^ 
cides exactly with .the continuous version of V 
the zero-one problem In the free variables. 
Hence, if in computing the coiiq>osite constraint 
.the partial solution is not fathoMiand if 
the dual variables are" integers (0 1), an 
optimal completion of the partial solution has* 
been found .and Jsacktracking can begin in the 
next iteration. . In . the computational exper- 
ience quoted by Geoffrion, It is stated that 
the "u^e of the imbedded L. P. gfeatly reduced 
solution times in virtually every case." This 
la opposed .to the use of the algorithm without 
the advantage of the imbedded X. P. ' 

The program is written in-general f oim and inakes " 
no advantage of special problem' structures. At" 
each iteration a sliq>le test for binary infeasi- 
billty and for .'conditional binary infeasibllity 
is . performed, with each constraint . be liig. consi-* 
dered individually, . « > . ■ 



. -cons trklnts .carried. ,The L.P. routine lised employs 
the revised simplex- method with- explicit * inverse 
^ by, Clasen [4]. ■ . . . 

; ■The, only difficulty encountered in using- this ^ ' 
code concerned the ,value of 2KBAR input (0) . The 
program. looks, only for feasible solutions with 
value at least .99999 less than .the best 

feasible .solution. However, on the IBM. 370/168 
computer^ the. code originally did not find the 
optimal solution on many problems, but terminated 
at a solution which was always 1 greater than the ' 
optimal solution (all c^'s* being integer) This 
difficulty was easily reijiedied by setting ZKBAR 
equal to -0;5 on. the input card. This difficulty 
is felt to be^ machine dependent. 

Two versions 6f RIP30C were investigated: one in 
which the variables .were input in the -ord^r given 

.in^the problem (RIP30C),' ^d the other in whidh 
the variables were initially sorted in increasing ' " 
order of c (SORTED .RIP30C). Thlsr latter refine- ' - 

^.ment has' b^en found. effective telsewhere, and Is 
easily implemented by the user with no modifica- 
tions, to the existing code required. ... 



D2IP1 fLemkg and Spielberg ri8l) 

_ ^The code of Lemke and Spielberg employs a direct 
search technique developed by the authors. On. each 

' forward step, one variable is elevated to one^ on ' 
each backward ' step one variable: is "cancelled" 
/bade to p. Since the algorithm begins with' all 
variables at' level 0, the algorithm terminates (in ^ 
a formal" sense) when all variables have been "can-, 
celled" back po level 0. 

At each partial solution, the ^algorithm resolves 
the O-l^subproblcm in the currently free variables* 
A baclcward^tep is taken whenever it can be ^stab- 

• llshed that any permissible forward step ileads to ^ 
an unreMvable infeasiblllty or to a feasible 
solution not better than the best solution thus 
far obtained (ceiling criteria). On forward' steps, 
a "preferred set "of the k free variables is .con- 

8 true ted by using special types of Gomory cuts 
and a process termed complete reduccion^ The. pre- 
ferred variable yielding the maxlmal^^Balas, value 
la then elevated to 1, When the sxxbproblcm^in . 
(k-1) free variables is- resolved, th^ last variable 
to be elevated to 1 (LIFO procedure) is cancelled^ 
to level 0,* and ^ the subproblem in the remaining 
(k-1) free. variables is again resolved* The 
authors state that the procedure "of considering ^ 
a subset of the free variables^ the preferred ^et, 
- reduces substantially the computing time required 
and the number of points to be considered. 

The code uses all Integer arithmetic i^lch has a 
ver3r distinct advantage: half-ward Integers can 
be used for the arrays > resulting In a larger 
.problem residing In core or primary memory 

The^program performs a change of variables (if 
: necessary) from -"problem" ta> "program" variables 

id which 0 i c c • - . Thlsy -as well as_bther 

elaborations to the"^ basic algorithm, has^the'; 

effect of significantly reducing computation time. 

The authors state that the code often finds the 
' optima ly or at best a feasible solution rapidly,' 

with a major portion of the computing time being 

expended in the "clean-up" phase,' where the optl- 

mallty of the solution is verified. 

PZLP (Salkln and Spielberg r24.1) 

The code of Salkin and Spielberg employ s^a direct 
search technique which is an elaboration of Balas ' 
additive algorithm. In addition, the code pro- . 

• vldes for the initial origin tb be. generated- b y an 
imbedded L. P* -roundup start, and also provides for 
the search to, be restarted at improved zero-one 

solutions. While restarting the search neces- 

sarlly Introduces enumeration redundancy, the 
authors state that "the length of the search Is 
Inversely proportional to the, closeness of . the 
origin to the minimal solution" and the- redun- 
dancy Introduced, by restarting may be more than ' 
compensated for computationally by' the :reduction ' 

In time expended in* verifying the- optlmality of 
the '"solution. Comp;atational results reported 
empirically verifies this contention f 23 J . The 
«ode also contains Infeasiblllty, cancellation, - 

• and celling criteria similar "to. that of DZIPl... 

The code uses real arithmetic, and, in addition, 
contains some double precision arrays and varla* 
;blefr. The program in .the form received required 



a few minor prc^ ramming chai;iges to run on the IBM 
370/168 computer. Along with a few "carriage 
' control" and other format statement changes, the 
major change involved correcting the program from 
returning a "no feasible solution" termination 
■"^:when. the first call to the imbedded dual^-slmplex 
'routine yielded the optimal solution to' the zero- 
one integer problem. Apart fram these minor prob- 
lems, no additional difficulties were encountered - 
in using the* program in its original form. 

HOICOMB CHolcomb ClSl) , " . - ^ . - 

j^' The program written by Holcomb at Union Carbide 
is a variant of the Balaslan algorithm with 
several Jieuristlc tests available for infliiencing 
the search strategy. These heuristic tests re- 
place existing tests in the .algorithm and allow 
t}:|e user to select^'a procedure for conducting the 
search. Heuristic options available include pro- 

^ .visions for initializing the program with a start- 
ing solution which satisfies a 'Vorst" constraint 
. or which satisfies' a surrogate constraint .which 
is the sum of^all constraints. Other^ options 
available include -a provision for ent^ripg. ♦-more 
than one variable into the basis in an fteration • 
^ and for raising to '1 the first eligible variable 
rather than conducting .a^ time-consuming ' test to" 
determine thc^ next entering variable.' Two pro- : ^ 
gram strategies are available which are used when- 
ever variations are present in the magnitude of the 
ratio ^j^^ij^^ whenever some constraint coeffi- 
cients a^j are such that Cj - a^j > 0. 

Best results were obtained using" this code ta solve ^ 
the Included problems by allowing for more than one 
variable to enters the' basis in an .iteration and 
by p^widing the program with a starting solution 
which satisfied the sum of all constraints. 

The maJojFity of the arithmetic calculations per- 
formed by the code are performed in the fixed-, 
point mode*. - 

^ ENUMBER8 (Trotter [29 » - * - ■■ 

' The ENUMBEK8 computer program was. developed by . 
Trotter^and Shetty [28] to solve the bounded 
variable ■ integer programming problem using impli-. 
. . cit enumeration techniqiies. .Their approach repre-, 
sents an.1 extension to Geoffriotfs [91 algorithm 
for the pure integer problem, employing simple 
fathoming tests based on: optlmality and- feasibility 
considerations, and ezrployiiig surrogate constra^ts 
— -similar to those defined by Geofffion. The authors 

'also employ techniques, to tighten the bounds -on 
free variables- using the" L.T. based procedure 
coutained in the^ a^lgbrithm, the effects of which 
are to simultaneously fathom several partial ^ . 
solutions- at a. time. — , ' 

As a itser option', the augmentation phase of the 
algorithm (explicit enumeration) may be guided by 
two mutually exclusive criteria. (1) The maxlmiim 
BALXs value is used to select that variable x^ 

to elevate to d, or its current upper boilnd'. (2) 
J 

The variable x^ which minimizes dj for all free 
variables is fixed at its lowest permissible value. 



The authors state that "utilization of the bound 
redefinition capability appears to offer a general 
• Improvement in the -per f 02™ nee of the algorithm 
regardless of the augmentation rule used» although 
in comparing the computational experience obtained 
by varying the augmentation rule» the results are 
less well-defined* 

Several difficulties were' encountered in using 
the ENUMBER8 computer code on the .IBM computer. 
These difficulties arose, for the most part,' 
because of 0-subscripting iii the arrays/ with 
concomitant problems with several of the indices 
in DO-loops (betog out of range) . ^ ^iSeveral 
changes thus had to be made In the program, and 
it is indeed possible that changes made in order 
to run the program on the ^IBM computer may have 
affected the logic originally intended by the' 
authors. Unfortunately, time did not permit us 
to flow-chart this procedure to examine fully the 
implications of all changes made to their pro- 
cedure. We have since used ENUMBER8 on.. a/CDC6500 
and a UNIVAC 1110 computer,, and have not cxperir 
enced the difficulty we encountered on the IBM 
370. < . , - 

ENUMBER8- is used in these experimcats^ to assess 
the efficacy of Trotter and Shetty ''s4pproach for . 
solving the pure binary problem Cd *1, all j), 
and.for con^aring their direct approach for 
treating general integer variables with a binary 
procedure using a binary expansion to represent 
the integer variables* 



All codes were tron^ lied into object decks using 
the FORTRAN H compiler with optimization feature 
(OPT - 2) in order to obtain the lowest . execu- - 
tion Times possible. TheVtiming subroutine for' 
each 'code returns relative CPU time in mil 11- 
sectonds usisg' the "T-timer" macro available 'on 
, the supervisory -system. 

Computational Experience. 
Cap 1 1 a 1 Budge t in g P rob 1 ems 

The capital budgeting problems given in Table 2 
were orgi-nally solved by Petersen [21] In his 
Investigation of variants to the basic Bala s Ian 
algorithm, some of which were suggested in an 
earlier paper by Glover. The problems vary in 
size from six variables and -^ten constraints to 
. fifty variables and five constraints. 

" , ■ '. ^ . . ' ' " 

The 1 we St overall execution time on these seven 
problems was achieved by SORTED RIP30C, followed 
by RIF30C and then by DrLP. Both lyziBl and 
HOLCOMB began experiencing difficulty in.soljring 
these , types of problems, in the 25-30 variable 
range and above. Apparently*, the addition of 
the start and the adaptive origin concept to 
the iHeas originally developed by Lemke and Spiel-* 
berg possesses great merit in solving these types 
of problems as is shown in* the computation results 
for D2IP1 and DZLP in Table 2*. (D2IP1 and 
HOLCOMB were unable to solve the seventh and 
largest capital budgeting problem within the time 
limit specified'..)- Sorting of problem variables 
was also of sotae "importance ^in reducing execution 



time for RIP30C. Unforttjnately^ employing 
ENUMBER8 with all d'-valuca^ set equal to 1 did not 
result in low execution tls^s for this group of 
problems, especially as the numbejr- of variables 
in a problem increased. This corroborates the 
experience reported by Trotter and Shetty [28] In 
solving purp binaryproblems with their procedure. 

* Single-Constraint Allocatldu Problems 

Nine allocation problems were formulated by 
Trauth and Woolsey to investigate the sensitivity 
of integer programming algorithms to minor changes 
in the'problto matrix* Each of the problems conn 
sists of ten variables. The problems differ from 
One another only in the value of b» the right 
hand side. All codes were able to solve these 
problems in less than -O^ 10 seconds per problem, 
hence the results arc not presented in tabular 
form. ,. , , ■ 

IBM Test Problems - . V . - ^ - 

The XBM test ^problems 4re a pot-pourri tff integer 

programming problems i^hich (except for 'problem #3) 

feature jnatrices of O^s and I's. Trauth and - 

.Woblsey selected these problems for examination 

because of "the wide disparity of. solution times 

between various approaches and: because of the 

•multiplicity of optimum integer solutions.*^' DZIPl 

and HOLCOMB experienced difficulty in solving this 

class of problems when the number of variables' 

■ increased to 30. The ENUMBER8 programming code 

recorded its best results when the minimum branch 

rule^ was used for variable /augmentation X these j' " 

integer problems were treated dire&tly by ENUMBERS). 

Overall best resdlts^were recorded by the "unsorted 

version" of RIP30C and by D2LP, as indicated in 

■ Table 3. ■ . , r ;r .. ^ . ' 
■ V- ■ 

^' IBM problems and 5 differ^- from each other only 
. in the b-vector (bS t^) ^ yet each of the codes 
examined experienced much Tnore^^difficulty in 
solving problem SL^than. in' solving problem 4. While 
the difference in solution dime's cannot be explained 
by the coii5>utational results reported, one can con- 
jecture that constraint severity (as determined by 

m n a^ 

Cr Cr*'tr'5 5/°^ ^ these structured problems has 

i-lj-1 ^i X 

as pronounced an effect on solution times as do 
th€-nt;ttiiber of variables and/or the number of con- 
. straints. The results reported in Table 3 are 
similar to those reported by Trauth and Woolsey. 
p7 , p- 491, Table V], except that each of the . . 
codes showed less susceptibility to the large 
number of constraints in problem .9 than did the 
integer programming techniques they examined^ 

Flxed^Ch_arge_ Integer Programming Problems 

The fixed-charge integer'' programming problems of 
Halrfii were formulated as'' zero-one iijteger program- 
ming problems by a binary expansion of the bounded 
' integer variables using the binary procedures 
. examined, and were treated directly with the 
EITOIBER8 program. Each of these problems features 
special, constraints which force certain variables^ 
to assume nonzero values if other variables take 
on nonzero values. Despite the small size of these 































T*bie 2 








. ■ ■ 




' ■ . ' .S 


"I . * 




Solution TiaiiftB for ^etarsen 


'a" **• ■ 










>* 

< ■ 

\ 




' Capital Budgeting Problems 










Probl«a 


Suaber o£ j 


. ,^umb«r of 




Solxitton 










0-1 VAriabXaa 

. ■> ■ ✓ 


Coa«tr«lnc« 




SORTED 
RIP30C 


DZIPl 


DZLP- 




> EKUMBER8 
V BALAS 


■ t HBR 




6 


10 


0*050 


0.040 


0.077 


0.040 










10 


10 


0.073 ' 


0.077 


0.090 






U • X3/ 




..' 3 ■ 


15 


10 


0.174 


0*110 


0.340 


0.130 








4. 


20 ' 


lb 


0.223 


0.137 


1-727 












28* 

- ^ 


10 








1.l770 


8.407 






..6 


39 


5 


1-706 


0.947 


46.984 


2.337 








"7.' '•. 


50- 


5f 


" 3V587 


\ 2.780 


^150* 


15.917 




29,167' 


31.3j50^ 


Simamxy - 


Nixiaber of ProbX«as In Which 
OptlMl Solution U«« 7ouz>d 
and Verified. •• ■ ' . : ^ 


. 

i ' 

J . 

^ . ■» 


7 r 


.■■ 6 




' 6 


7 


7 


"it, »• 


.»■■-. ■ ** . ■ 

Avensc JSolutica Xiae 
for Problems Solved 


.885 


.621 

■ 1 


12,541 


2.946 


7^422 - 


6.691 


7.200 



r • - ' . . ^- ■ ^ . . 

Indicates optiul solution was not found %ritbin allotted tlas of 150 seconds 

^IBX 370/168 CPU tla»^ la seconds 

I»d£cAtes sll Cj*« i^ave been aalt:is>llsd l>y 10 la this version of the problem la order to have all Cj-values la 
Inte^air fomCto aaka a Talld cooparisOQ beciieea coopoter codes)* 



problems (in terms of the number of original Inte- 
ger vaariables and number of constraints), they are 
quite often difficult to solve xising known 
approaches. Solution times ^or each of the fixed- 
charge problems using each of the programming, 
codes Investigated are shown In Table 4. 

^ The lowest mean execXition time on this > series of 
problexns was recorded by DZLP, followed by ^ 
ENDMBER8 usixi^ the • min imum bra ach augmen ta t ion . . 
'' rule, then by SORTED RIP30C^ RIP30C, HOLCOMB, 
DZIPl, and ENTJOTERS using' BAIAS' rule for .aug-^ 
mentation. The pUre binary programming pro- 
cedures not cn^loylng any form of linear pro- 
gramming to fathom partial . solutions generally 
showed a larger increase in computation times 
as the number of variables in a piwblem in-", 
creased, than did their. LP-based counterparts. 
■Also, the ENDMBERS program with the BALAS aug- 
mentation rule was not generally very effective 
in solving these, problems. . 

It. is interesting to compare the results in Table 
4 with tH6 integer programming results of Trauth 
and Woolacy [27, p» 490, Table III] on- this 
series of problems. Problems 1-4 and 7-8 are 
qid.te similar, differing primarily in the value 
of the b-vector and the value of the "fixed- 
charge." While the codes RIP30C, DZIPl and. 



HOLCOMB each record similar computation times /for 
problems i/ithin each of these two groups and a 
marked difference in computing jtimes -between the" ► 
two groups, all were able to solve the problems 
in a reasonable amount of time*,- Of* the five 
integer programming techniques- examined by Trauth . 
and Woolsey, only^ one was able Vto solve , all of * . 
these problems within the numbctr of iterations, 
allowed. ' Considering the difference in operating 
speeds of \the^coiiq)uters used, it would; appear that 
problems 1-4 vere solved in less time by integer 
programming. However, all binary programming 
codes were able to solve each of the fixed -charge 
problems, and each' showed far less variability . 
. in solution time between the two groups of prob- 
lems, 1-4 and 7-8. * 

In order to compare the direct approach of - 
Trotter and Shetty^ with a binary expansion of 
the integer .variables and a 'pure zero-one approach, 
the fixed charges and the right hand sides for the. 
problems listed in Table 4 were multiplied by 10 
and then by 100» For tt^e binary procedures, this 
resulted in a maximum pr^|^*cm' size of 71 variables; 
for the Integer approacf^^this simply resulted In* 
Increasing the bounds on the variables, d . 
Summary results on these larger problems are 
prctsentied'lLn Table 5. JDue'to the superiority of 
the minimum branch rule iri,jthe ENUMBERS program 
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in solving the original version of these problems, 
this was- the only arcernative investigated for, ^ 
these larger problems. - . ^ 

Again D2LP recorded the lowest, execution times 
for both problem sizes » and. in general, those pro- 
cedures not using any form of linear programming 
began to experience -significant difficulty in 
solving these types of prohlems when the number 
of variables in the problem increased "beyond 30- 
It is also' Interesting that the use of. a^ binary 
expansion for the integer variables when used, with 
a procedure employing linear programming to 
fatbom par tial Volut ions generally^ resulted in 
as low if not Jtower execution times for this 
group of problems ' than did ^ Tdirect treatment of 
the integer variable^. The differences in recor- 
oded execution times' wije, of course, a, function 
of both differences in computer programming 
decisions as well as a cjifference in algorithmic 
approaches developed.* " ' 

Summary Observation 

The, computational experience described in, the pre- 
vious section reveals the efficacy of certain 
Improvexnents to the basic Balasian algorithm as 
Impleiaented by computer programs supplied hy the 
authors of the^ improvements. Any evaluation of 
these modifications is necessarily confounded with 
the programming decisions made to Inclement each 



improvement, as wel'. as with the programming deci- 
sions made regarding the coding of the basic 
algorithm. Neverth-t^r:::s , the data does indicate 
that certaiir^ Improvertierrs do indeed accelerate 
convergence on certrc^ir categories of problems. 
The data further provlces direction for the tiser 
of these techniqucn Cor .solving certain zero-one 
and. integer programming problems by 'existing, non- 
proprietary computer programs,^ and provides a 
yardstick for •asse^ssing ?the relative worth, of 
proprietary programs base^d upon', their cost. Addi- 
.tionally, the data sxiggests certain avenues which 
might be explored in devfe'loping future improve- 
ments to known -.enumeration approaches, 'Some of 
which have already appeared in the open literature 
■ > • I- ' ' ■ - . ' 

Imp^rdvements to Existing Techniques 

With the exception of perhaps the allocation prob- 
lems, it appears that any reduction in computation 
time which could be achieved through the use of 
integer arithmetic is more than offset by the. use 
of real arrays used in conjunction with an L.P.** 
routine to accelerate convergence. This is evi- 
denced by the lower overall confutation times 
recorded by SORTED RIP3pc, RIP30C and DZLP. It 
would appear, therefore, that any improvement made 
in the method of solving the imbedded linear pro- 
gram would have the effect of significantly reduc- 
ing solution times. Geoffrion and Marsten fH] 
discuss an improved RIP30C which incorporates 
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(among other changes) a duiil linear programming 
subroutine with column, generation -iyhich is purpor- 
tedly more^ efficient than the expl:Jcit inverse- 
revised simplex recthod' available in the current., 
'version ^f RIP30C^. Given the current state-of- 
the-art in enumeration programming, it seems safe. 
*to conclude that any improved program will neces- - 
sari ly incorporate some type of an imbedded 
routine,! although the choice of the actual L.P. 
routine is more^open to question. It is worth 
mentioning in th^s' connection thPt the problem . 
which led to thd discovery that DZL? may on ^ 
occasion return a "no feasible solution" when the * 
first call to the L.P, returns the optimi^l zero- ■ 
one solution was solved in substantially less tiine 
by RIP30C with the Clasen L.P. routine than by 
DZLP. 

In general, solution times were lo^^er for the pro- 
grams- which use a starting soluti^a^jiet^rmined by * 
solving th^ continuous version of the zerb-one. " 
■problem by linear programming, and then rounding 
% this, solutiion. Subsequent testing with both KIP- 
SOCand DZiP on the Petersen capital budgeting 
problems u^ng a null starting vector and a round- 
ed L.P. solution revealed the general ef f icacy of* 
initializing the implicit enumeration algorithm 
, with other than a vector of zeroes. This would < 
' tend to substantiate Salkin's [23] contention 

that a good initial .solution can s,ign.if icant^y . 
..-xeduce the length of the search and would further 
-^'^uggest that otner initia lizing, sdrfeepies be . 
.examined. Byrne and Prall [^3] :*have" reported sue* 

cess in this area, and the use of ^ a heuristic • 
* starting procedure such as the "effective gra- 
' dient" procedure of.Senju and Toyoda [25] also 
investigated by Wyman [31] shows promise both for 
initializing the implicit enumeration algorithm* 
and for providing a good initial bound on. the 
objective function value for capital budgeting 
type problems. 



Although less convincing than the general *ef3^Sacy 
of employing LP-b a sed routines, the data for the 
problems examined tend to indicate that the use of 
a binary expansion for the integer variables may 
be as effective' as treating the integer **variables 
directly, as least within the^. context of impli- 
cit-enumeration. Naturally,' the differences in-, 
'results reported is confounded with differences in 
programming decisions made to implement each, of 
the algorithms described.. 

Special Problem Structures .- 

Integer and binary ; programming problems with 
"special structgires" are currently .being solved^ 
in a modest amount of ^computation time for prob- 
lems involving a few hundred^ variables • ^Thangavelu 
"a^id' Shetty [26], for^ example, have developed a 
spebial purpqse enumeration procedure for solving 
a binary foMulation of. the assembly line'balancing 
problem, and q^iote impressive results;. Their pro- 
cedure has been generalized to the project and job- 
shop sequencing problem by Patterson and Roth [20], 
and again good results have been reported. Thus, 
it is possible that .special purpose procedures 
will,, in the future, replace general purpose 
algorithms on special types of problems- as ;nore 
icnoweldge is gained in solving these special 
structure .problems. - * ; 



■ Predicting the Time Required for Problem Solution 

>One of the more important formulations (by appli- 
cacion)of binary programming is the capital*^ 
budgeting problem. Although these type^: of prob- 
lems do possess a fairly predictable structure 
(dense matrices, .positive a^^'s, etc.) It is not 

" a structure which is easi'ly exploited. Interest 
thus centers on whether solut^ion times can be 
predicted as a function of variables o«Cher than n," 
t'he: number of .variables in a problem. Th^ experi- 
ence gained in solving the IB>f problems of Section 
III suggests that con-s train t severity as measured. 
- by the "amount _of slack present in a constra£nt" 
chn influence the -time required for -problem solu- 
<tion. 

, N^ine sets .of ten capital budgeting problems were 
computer genera ted. Each set contained 50,100 
or 175 variables and 20 constraints (this would 
correspond to^a planning horizon of 5 years of 
4 quarters each, or 20 years of one year each, ^ 
etc.) Coefficients for the problems were generated 
randomly such that problems within "a set had 
similar A matrices, but between sets differed in ' 
the v^ri,ability^ in c / and b^ values.* The^oblem 
generator is^nioVe fuily". desc-ribed in Ference £61. 
•An attemp.t was th^ macle to solve the generated 
-i.probleins with th^ sorted version of RIP30C. 
These, results ate summarized in Table ^.''.in 
general, those problem^ possessing the higher 
vai^ability in .b. values (for similar A-matrices) 
were solved in significantly less' cogoutation . 
time than those with less variability. S6bseqi;ent 
experiments with capital budgeting problems with 
11-75 variables indicated the most significant : 
variable. in predicting solution time over the ' 
range of problem sizes examined was aS variance 
measure of constraint severity given fey ' 



te (( z 111) - X sEv)^) ; 
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The R-squared values, in the regression models"^ 
developed were on the order of 0.88, indicating 
it is possible to identify classes of aqid charac-., 
teristics of problems "which are likely amenable 
to solution with implicit enumeration techniques. 

Conclusion v , - 

Pure binary and integer programming problems werie 
input to five different computer codes, each of 
whick incorporates various modifications and 
improvements to the basic Balasian algorithm. ^ 
•The overall conclusion is that the surrogate con- .• 
straint concept developed by Geoffrion and pro- 
gransned into RIp'SOC and the dynamic origin con-- 
cept of Sal kin and Spielberg are the most effec- 
tive of the improvements examined,^ although oth^r 
improvements were often effective on some problems. 
The effects. of determining gqod initializing 
solutions were discussed, and suggestion's for" ^ 
incorporating other improvements into zero- 
one integer programming routines were made. The ' 
results reported tend 'to show that certain . 
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\ .*■ Table 6 ' . . \' ' ' \ '^\'''' " / . 

Meaii- Solution. Times -Required to Solve Randomly Generated Capital' Budgeting Problems 



Nuiabet'of -Var±a1>les" 



■50- 



Variability- In'> - 
Cj -> values' 



Lov 



High 



Law 



. /High. . ■ 



High 



Variability.^ la 
b^ - values 



High 



Low 



High, 



LOV7 



High 



Low 



Hi 



'High 



. Meian- Solution Time 
Ten Problems' 



r 37.36 



7 

•^2.146 



36.49 



.4 ■ 

1.689 



150 



22.7a 



150 , 



9.947- 



38.89'/ 



IBMy570A68 CPU tlmfr, in seconds ^ 

Indicates no prcSlems were solved within tijae limic of 150 seconds 'p^r proVlem. 



types of pure-integer programming problems * ' , ' 
may be amenable t£ solution by zero-one integer 
programming.-., - J . .- . " ' 
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- 1- The special types of cuts employed in the 
Lemke-Spielberg" code (bziPl) ^ although termed ' 
Gomory cuts by the authors, 'are ^not the same as 
those traditionally referenced ^in the literature 
as Gomory Cuts., and used in solving the general 
IP problem (cutting planes)-. . , 

2. ^- The original use of the 'generalized origin 
tephp^^^ is due to Sjpielberg; "Plant Location 
with^Generalized Search ^Origin, !* Management Science -, 
Vol* 16, No. 3, (November 1969), pp. 165-178 



4 



One final comment made by the ^anonymous referee. 
Vas that the paper '^probably is not up-to-'date in Z*^'' 
the sense that all promising, new approaches are 
included. The intent of tfie investigation was to - 
• evaluate various "de'pth-f irs^J^r-search" appi^aches 
for solving^' the. binary and tfie bounded integer" *- 
prog'raimaing problems using computer software^ 'y-j 
(FORTRAN programs) supplied by the originator of 
the algorithm evaluated. Within the restrictions 
of the techniques evaluated and the general " ^ 

' availability of software for implementing the^e «, 
techni^es, the paper ^i)robably"is fairly up-to^ 
dat^ jsl though certain other search strategies may 
offer more promise in obtaining solutions rapidly. ^ 
Hopefully, the computational experience reported " 

. he^'ein can be used as £i yardstick for assessing 
the^ 'promising ne^J^pprocches'^^for. solving 
binary and' bounded, integer progranming' problems. 



Footnotes 

1. Investigations of the efficacy of various 
mathematical, programmii^g algorithms for other than" 
the- binary programming problem have recently 
appeared in the open literature. A computational 
■investigation of the ^*p^r a -integer" programming 
problem, for example, is reported by Trauth. and " . 
Woolsey [27]. Zionts^has, recently performed «. 
some empirical tests of the criss-cross metiiod ; ^ 
of linear programming [32]. lAnd Braitsch compares 
four quadr^ic programming ■ algorithms in a 1972 ' 
paper [2]. ■ ^Jiy ' . ' 

2. Each of ' the programs examined herein is " ^ ■ 
available through^jTIAKE or RANp Corporation 

at a very modest (usually mailing) cost; 

3. The improved RIP30C was not available for- 
tes ting, • 

4. Geoffrion^^S^ a private communication) reported 
solving cafpital budgeting problems involving more 
thaii two -hundred variables -with the improved 
RIP30C. ^ 

5y- r Eacl^'of the computer* codes ■ examined requires 
tiiali-cfertain input .parameters be set to .influence 
the 'direction of the search .and the search strategy 
employed. Hence any conclusions made regarding 
thec:efficacy of the various -approaches should be 
made .in light .of the version actually examined. 
Extreme care was. exercised in determining the 
strategy; to .be followed; author*^s recommendations 
' were usually adopted. . However, it is possible 
.that a given approach could record lower; solution- • 
times on' a:' given class of problems /us ing..a variant* 
of the. solution strategy /Examined. ' • /. 
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.Abstract 



, This paper presents the results of an 
experlmeat to determine the relationship 
between parameters ^ich can be used to describe 
linear programs and stsandard measures that can 
be. used to describe the performance of the 
r:Slmi>lex algorithm. The test problems are -gener- 
ated by LPCENR, a fortran subroutine that gener- 
ates linear programs, and solved by MPS/360. 
Problems, vary in size from. 50 rows atnd ^100 
col umns to I50 rows, ' 300 columns and 50 upper 
bounds with varying densities and solution 
characteristics. Correlations and least squares 
fits are calculated .to determine the relation • 
between parameters aiid performance measures. 
Several .unusual relationships are reported-. 



I. . Introduction ^ 

.. Several experimental: studies in linear , 
^- . -programming have. appeared -in -the literati 
^' ■/(see £11, [2]'» [5] , t;^), .but .much of . the experi-" 
mehtal/'worlt . is unpubl fished and is transmitted- . ' 
as'^ folklore. Jot exaraph^tt , is often stated ■ . 
\ ' tha^t., the number of iterations: is between one and 
three ■times the number 'of -rows. This study was , . 
initiated to examine the. relationship becween ■ 
: severaL parameters including, some often quoted 
■ ones and; some performance measures of ,the simplex ' 
alpgrithm. At the same time the pe3;^o^an.ce of 
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liPGENR: (see [31) could , be examined. 



j^^;^ Software 

LPGENR (aee [3] and [8]) is a fortran sub- 
routine system which generates LP problems with' 
prespecified characteristics and" a known optimal 
solution using a pseudo-random'number generator. 
Options allow the parameters to be passed or 
generated with a specified range. Further, 
. output may be obtained in several forms includ- 
ing MPS card imaW format. A brief /description 
of the algorithm J^r-genferating the- problems is 
^iven in..appendix .o^ie- 

.A ' The. gtenerated problems' were solved using 
TTtM.' 3 1^25/350 with no default changes on' the . 

-a 360/65 under the OS/MVT operating system. 
_ is later, proved; tp be a problem* since the ' . , 
aujLOCKSW- switch. war on. This switch causes .an 



IJiyERT demand to be controlled by the wall clock.' 
Since the operating system is murtiprbgrammedV- 
re invers ion was . per formed mor e f r equen t ly tjian " 
desired, sometimes, only three iterations apart 
and in a manner not compl'etely predictable. About 
16 problems were rerun and are presented in the 
appendix. . . - . 

The stj^^tistical analysis was performed: by 
the ' Statistical Analysis System (SAS) (see [ 7] ) . i 



. -3. /Eicperimeut' . ^ 

The parameters chosen - for study were: - the 
number of rows varying from 50 to 150> -^the' number 
of columns varying f romr 100: to :300, -the matrix 
density varying from about .40 to .04, the number 
of- upper bounds varying from O .^o half the number 
. of rc>ws;.* Further, various "problem types were " 
, 'included ttiat defined the. clraract eristics of the 

optimal solution*.-' - -'^^^ '"'^ ' ' \ 

. , ■■. " : ., -'.^ ' -■ '\ : . >: ' 

The optimal so lutio'h characteristics for 
this study can. be divided, into, three 'ciasses^: '* 
the- type of optimal- solution, , the^ size of' thef, - 
/reduced Jiosts at optimalityi and the.* number of ^ 
. variables at their upp^ bound. ■ Th'ere^.werfe ..three 
. optimal solution types: degenerate, unique^with 
all basic variables greater than zero, >and alter- 
'native. For degenerate solutions the number .of ■ 
■ basic variables at zero was set to be twetity-ftve 
^percent of the numter ,$f ifisws.' - -For alternative 
optimal solutions .^e'^ number of -rionbasic"' columns: 
wi th^ reduced , cos t s of zero -was ^e t . tq^ be ten .% ^ 
percent of the ntjinber of rows. The -magnitude of 
the.* reduced costS& '^of nohbasic columns at optimal- - 
ity are spe'tified by raultii).lying tfhe objective 
coefficient which would produce a zero reduced 
cost by the factor, 1 7 r,., where r is a uniform 
random variable over the "'interval 0 to either 0-1, 
0.5 or 1.0.. The number of variables at bound "in 
the optimal solution 4;as set. to be half the 
number of upper bounds. ' 

' All other input parameters for *LPGQJR were 
*held. constant. Each problem* was. *a 'maximization 
with ail equality rojjs. The nonzero entries in 
the constraint matrix wer;^ uniformly distributed 
between -2 and 10'; the-optlmal nonzero primal and 
dual solution values from which '\he: cost row and 
righthand side are generated were uniformly 
distributed between 0 and 20; ajjl Values were 
rounded to. the nearest integer. ' " ; ; 



. ./The performance measures recorded for each 
• test- problem vere the .number of phase qne itera- 
. tioas,. the CPU. time, cj the execitibn st^ and 
the percent: by vhich. me generated opfioal value • ■ 
deviated from the calciin^ated optimal -.value. The 
/CFO time includes a- modest amount of time for 
. .setting up . the problem.. ■ ' ■ 

■ It was hypothesized :that .problems in which,- 
npnbasic variables at optimality had large 
reduced costs vould solve faster since the , 
op tiinal. columns , might"' be easier f<ir^the algorithm 
to choose. 'Also, probleias with alternative^, 
'optimal solutions wou I'd be easier^to find 'since * 
"the algorithm "need on,Ly_find one from many. It 
is obviou^that the ..density has- an effect on the- 
number' of computations,' but'-little has. been said 
concerning its 'effect on the number of iterations* 



" ■ , * • ' 4." Results 

■A total of 239 problems were solved and the - 
^ results are listed in: appendix two,- In -29 cases, 
tolerji^^fehecfes- made after reinversioft caused 
the al^HKm to terniiaate. In all but three 
cases, TEse "can be 'recognized by a nonzero 
value in the last co-lumn. of the table* . In five, 
cases (two of ,the f ive are replications!) indicated 
by a * in the, table, the solution was in' fact ' 
' optimal but in each case the'^um 'of infeasibil* 
v itles-vas zero and- several degenerate pivots were 
.. taken before "the problem .was declared Irifeasible 

after the tplerance -chTecK.' Some problems, that 
' were rerun ^fter tolerance checks declared them 
infeasible, con tinned- passed 'the'.point \diere they 
were previously terminated after reinversion and 
' subsequently, \the. optimal solution was found and 
. declared .as- such indicating 'that in a* sense the 
< problem ^is^se-tt^correcting.-.'" 
\- * "^V ^ V . ./ 

. Least' sguares ''fi^s and correlation coeffi-^ 
"cierits were calculated for the * 50' and .100 row 
' • pro"blj^s.;that ;,r:eacTTed 'ith The 
problem^^ith 150 cons"traints were not inciuded 
.in- th-ese calculations sino^ -ohiy three reached 
^pptimalfi^ and the others encountered jaiimerical. 
probiecS?!? The correlation coefficients .-are' Listed . 
in- table one. < Least squares; fits were calculated 
-with^phas-* I iterations, phase' II iterations, ' 
total iter.-.cions -and CPU time as dependent-' " 
'variables. The calculations^ are p"resented for 
CPU time but not discussed because of the wall 
■ clock reinversion demand: " All 'fits were'^forced ■ 
*to have a zero intercept. The results are 
presented in tables two, three, four and 'five. 
The .values in each row are the least squares " 
"'coefficient :ot the indepesndent variable.and " 

standard error of "the coefficient. :The last 
t three variables are the coefficients for the 
/ degenerate, unique .anS alternative optimal 
solutions which were created as dummy variables. 



."When total- iterations is the dependent ' ' 
- variable, the three variables contributing most 
to the fit are, in'order, the number of columns, 
the .number of rows and the density. The -coeffi- 
cient of the number of rows ''is 2.09 which is. 
.r^pv9^-istent with folklore. The number of 'columns, 
fc KJ v> independent variable ii^iich contriButed most 



to the^it, .had a coefficient of l.f. and the .. . ' 
density had a large n^ative coefficient. ... ' " . 

• [:-[:' * . « 

■ When .pbfase* o.ne iterationiJ^ is the dependent 
variable; the three variables contributing most 
to the fit are, in' order, the number of rows-, 

- the number of columns, -and' the density. The 
number of fows is more, important in explaining. . 
phase one iterations! since the criteria Is 'simply 
to f ind, ia feasible solution. Again the density r 
produces a negative influepcc^ " j ' 

■ -When phase two i^atidns- is the depejpdentV ^ 
^variable.• the three ^iables'contributi^ no'st . 
J to the- fit^ are,:, in. order,' the'number of coliinnsj- ■ 
the density and the number of upper bounds. ' ■ 
Surprisingly the number, of rows has a .negative - ■ 
coefficient, but "contributes very little to the • 
. fit, indicating , that :its.. effect is virtually 
negligible- The major part. of the expjq^nation 
is due to the number of columns and the density 
which again- shows up negative v ■ 

. - . ^ - ' -. > • . ••».■.'.■ 

Although the. dummy variables that represent 
the types of ;^ptimal solution, are nat among the / 

- variables cox^ibuting most to the. fits, it/is • . 
interesting, to note, some general trends. When 
comparing the three solution types it is per- 
haps easiest to consider the. differences between 
the -coefficients! In each fit:^Ising iterations 
the 'difference between the "coefficient^ for 
degenerate and either unique or alternative- ^ 
solutions indicates that degenerate solutions., 
require more f^erations than either '.unique or -: 
alternative .optimal, solutions- 



7^ - ^ 5. . Discussion of V the Results- - ': ■ 

The most surprising -results concern ".the" * 
■reduced cost perturbation and the/dehslt^ I^e-/ 
different reduced perturbations >ro^iuced Virtu-""' 
ally no 'effect on any of the 'performance. -measures 
in any;of the statistical calculations", / Another 
surprising result is the negligible eYfect of " 
the number of rows on phase two iterations.' - ^ 

Althou^ the- density Ijad a positive ' ^ " ' 
influence, on CPU- time," it produced/a negative ' * 
ef fect/^njthe iteration measures* ■ 'That is^ 
lower density .proM ems required' more - it^^ tons' ' 
than problems witlrhigher density. A problem 
with- lower density 'could "result in more -d^^ner- 
ate . pivots. The increased iterations -when 
problems have degenerate optimal soltitions could 
also. be the result of .a greater amount of degen- 
erate pivots on the way to the optimum. Both 
of these speculations support the' folklore that. / 
degenerate problems often, require more itera- 
tions. ' 

• To our knowledge this, is • the ' first:', study, 
that has'.attempted to consider the joint effect 
of a number of parameters. A question not 
answered by this study is ^how generated problems • 
compare' to real world problems. One concltision 
that can ie made is that the generator can, be -.^ 
specified to create problems that confuse the 
algorithm* For example, the problems , that were .• 
-declared infeasible when they were optimal*,. 



;. 

/■>'-• 



Correlation Coeffiolents 



Qcmbcr"^ upper Ijoutirfs 



^ -Qumber ib^f columns 

''- dfcosity 
eS 



- ■ %A . t • ■ 
reduce^cbs't perturbation 



Phase*;'! 


: Phase 15 ' • 


Total 


• : CPU 


iterations 


. U Wt^oq? 


Iterations 


^time 


.49 ' ■ 


-.31 


713 ■ 


.42- 


■ .34 ■. ■'■ ■ 






• 35 


■■ .13--''-; 


■ /.04 


-10 


M2 


; -;.38 ; • ■ 




• -.33 . ' 


.01 


.02 / 


-.01^ 


..00.^ 


• 00 



table oae 



Dependent variable: phase one iterations- 



-93 



iiideTyeddent variable 
nianber. of - rows ; . ■ . 
.nmiber '^of . columns^ " 
t&i2nber;tof upper bounds 
density ' . • 

reduced cotc p^%:urbation 
diegenerate optimnri' , • 
unique opt ixBri^ ■ . " 
al^eroative.optlnun: v. 

\ - . ' - table tw^ 



Dejpendent variabl.e; phase two iterations 



. R - 



.91 



table . three 



coeff ; 


std. err. 


independent variable 


coeff . • 


stdr err. 


^2.31 


0.16. 


number of. rows . . ■' 


, -0^22^ 


0.12- . 


0,55. 


■/ Wo. 04 


»; number.' of dotumn2? ' _ 




;. .0.03 


-0,14 " . 


6.20 ' 


number , of upper bounds ' • 


. 0.32r- ' 


' 0.14 




■ '"SUB ^ - ' \ 


density,' - ' , /" ; 


-106' V 


. • 22-8: 


2.13 • 


8.82 • 


reduced cost perturbation 


-2.90 . 


6*37 • 


-24.8 • 


18.8 


degenerate optimum 


2.74' 


13-6 


-50\7 


18.4^ 


unique optimum 


-20.9 


13.3 


-43.1.. . 


18.7 ^ . 


altemativfe optimum ' 


>-20.5 


■ 13.5 



1 



Dependent variable: total iterations ' '.^ 

' . \ ■ V ' r"^ - ,93 

Independent variable coeff * std. err. 

mmber of rows 2-09 0;2'6* 

nxSnber.of columns 1.30 0.07 ' 

number of upper bounds 0.18 0.31 

density: V -296 49-8 

reduced cost perturbation ^ -0.76 , 13.9 ^ 

degenerate optimum -22.1. 29.8 

.unique optimum -71.6 29.2 

alterxxative optimum -63.6 29.6 . 



Dependent variable: CPU time . " \ 

' : : V : '* - ••84 ' 

Independent variable coeff- std_.__err^ 
number of rows * 2,27^ 

number-of columns 0,60 0105 

number of upper bounds - 0*005^' _ 0.21 ^ 

density . ' 106 34.2 

reduced cost pei^turbation A<89 . 9.57 * 

degenerate optimum -160 '20,4 

unique • optimum ""177 20.0 

alternative optimum .■ -159 . 20,.2 



tabl^ four 



'table five 
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cx 

Ax - b.:: -;/ 

■■'O < 'x < .d ' 



LPGENR generates c. A, b and d ,as follows: ' V • 
step 1: Ge'njerate X > 0, an optimal -solution 
J , ■ - ai^^^^ V*) > 0^ an optimal set of • 
. . ^ ; mu.ltipliers. A degeneriaite' optimal 
■ \T solution will have more than, n-m 

■ " . variables in x at 0; a unique optimal 
■ solution^will have exactly m ^^zariables 

- . grater: than zero; an optimal' so lutipa 
■■"^ ' I - ^ith alternative optima. will have- more 
■,than,m variables greater than zero. . ' 



- ■ " ■Step-2;' . for i -1* to'n 

if -y*>0/ dj p x" 



■-;;if T^.-O, -d^ 



1 ■ 

:* (1 t URV[0, .5]}' 



step. 3:' for-i,*^;i;to m.-and j « 1 to n 

- if URV[;0, l]< jdensity, a., = URyf-2,10] 
: otherwise', a. = 0/ ^ , 



step 4? ^for i -"'1 to m 

■ - ■ b = A X*' ' ' 



step 5: for j » 1 to n . * V 

if X* > 0, Cj ^ u*Aj + v*. ■ \ ■ ; 

- if X* - 0, c| * (u*Aj + r*)(iiURVi;o,t]) 



J J, ' J J * 

where i is 'determined by the' sign, bf 
u*Aj + y* and t is the reduced cost 

perturbation. 



Note: URV[c,f]is a uniform pseudo random number 
between e and f. 
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NUMBER OF ROWS 



NUMBER OF COLUMNS 



NUMBER OF UPPER BOUNDS 





5C 




100 






25 






REDUCED 


OPT . 


ITERATIONS 




CPU 


PERCENT 




COST 


SOLN. 








TIME 
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1^3 
193 
159 
159 
159 
162 
•162 — 
162 
212 
212 
212 
201 

-201 
157 
157 
157 
-^01 
201 
201 
194 ^ 
19^ 
194 
120 
120 
120 



205 
225 
1 79 ■ 
185 
175 
179 
153 
-156 
143 
225 
166 

. 157 
193 

^ 158 
160^ 
: 167 
156 
167 
151 
150 

. 461 

.ri5 . 

^•133 
127 : 
157 
142, 
144 



398. 
4 1.8. 

' 372 
344 
334 
338 
315 
318/ 
305 
437 
378 

.369 
394 
359 
361' 
324. 
313 
324' 
352 
361 
362 
309 
327 
321 
277 
262 
264 



101.9 
101.6 
91.6 
133.1 
126.7 
131.8 ' 
166.7 
170.9 
1^2.2 
104.4 
66.6 
97.2 
137.1 
11 7. a: 
125.7 
1.69.2 
1-73.5 
177 . l ; 
831.4 
81.8 
■-^:e7.1- 
116.9 
: 117.4 
1C9.0 
l«>4 
151.3 
-153.0 



PERCENT 
OEVIATICN " 
FRCM CPTIMUM 

: 0.^ 

o..b 

0.0 
• 0.0 

. o.o - 

0.0' 
0.0 - 
0.0 
0.0 

0.0 V 
0.0 ' 
0.0 . 
0.0 ■ 
' o^o* 

o.a 

4.0 
0.0 

.' 0.0 

owo 

0.0 

' 0.0 . ■ - 

0.0' '-''^ ■ - ■ 
0.0 

o.o> .. 



100 



100 



: ReOUCED CPT> 
COST SOl.Ni 
DENSITY P£RjUAe« TYPE 



ITERATrOMS. ...CPU 1. PEPCENT 

PHASE 1 -PHASE 2 "tDTAL ; (SEC) Fa^^^ 



0*C6 


0 • 1 




1 


0*06 


0*5 




1 


o.ca 


l.C 




1 


0* 17 






I 


Om 17 






1 


0*17 


1 mQ 




X 


0 .33 


0.1 




. i: ■ 


• 0 • 33 


■ ' 0.5 




I 


0 #33 


1*0 




1 


OrnVB 






0 ^ 


0»0S 






0« 3d 


1 •o 




0 


0*17 






0 •■ 


0*1 7- 






0 


0*17 


1«0 




0 


C.33 






0 


0.33 






0 . 


0#33 






0 


C,3J 
0.33 






0 . 
0 • 


0.08 


O.l 




2 


0.08 






2' 


o*ca 


o.s 




2: ^ 


0«C3 


1*0 




2 


0.17 


0.1 




2 


0* 17 


0*5 




2 ' . 


0*17 


1.0 




2 ■ 


0.33 






2 


0'*33 


0.5 




2 


©•33: 


1*0 


-0 


2 



165 
165 
165 
1 80 
160 
180 
187 
187 
" 187 
165 
166 ; 
165 
186 
^186.* 
186 
176 
175 
176 
- ^.S 
?76 

* 181 
164 * 
• 164 
164 
■ 179 
179 
17^ 

■ 163 
163 
163 



-J 



C 
C 
0 

- 0 
0 
0 
0 

■ 'C 
.0 

•8 . 
13 

■ 7 
6 

. 7 
7 
C 

o . 

■ 0; 
c 
c 
s 

c . 
0 

c 

0 

c 

o- 

0 

c 

0 



165 
'165 
165 
• IfiC 
180 

rao 

187 

.187 
187 
.173, 
17^-9;: 
172 
1'92 

193 
176 ' 
175 
176 
175 
-176 
186 
164 
164 
164 
179 
179 
. i79 
163 
163 
163 



59.3 
65^1 
. 64.6 
93.3 
94.9 
93.5 
112.7 
143.5 
148.1 
59*6- 
61 .9 
"59.6 
M3.5 
120.1 
■ ICS. 4 
139.9 
129^5 
140!U4 
128ad6 

- 

55.5 

59.7 
r 60*6 

59*4 
1 12.6 

92.8 
1 17. 1 
142.9 
146.8 
1 4^.8 




NUM8ER OF RQWS NUMBER OF COLU*(NS / NL«BER . 01^ UPPER BOUNDS 



100 



100 



.50* 



DENSITY 

<^ - 0.08 
0.08 
0:08 ' 
0. 17 
vV 0.17 
? 0.17 
0.33 
,0.33 
0.33 
0.08 
0.08 . • 
^.08* 

0. 08 

01. 17 
0.i7 
0. 17 
0.33 

<r.33 » 

0.35 



REDUCEC 
CAST 



0.1 
0.5 
t.C 
0.1 

o.s 

1.0 
0.1 
0.5 

IWO 

0.1 

0*1 , 

0.5 

1 .0 , 

0.1 

0.5 

1.0^ 

0.1- 

0.5 ' 

1 .0 



CPT. 



ITERATIONS 



). TYPE 


PHASE 1 


PHASE 


2 total' 










I 


• . 147 


8 


ISS 


1 


147 


9 


156 


1 


147 


13 


160 


1 


176. 


21 ' 


197- 


/ 1 


177 - 


7 


484 


1 


177 




• 18^ 


1 


176 


■'7. 


163' 


1 


177 


' 3 


180 


1 


176: 


9 


185 


0 


149 


, , 3 


152 


C 


• 147v ' 


V- ■■ *a 


155 


0 


.'147 




161 


0.. 


1*7 


• - 7 


154 


. ■ ; 0. . 


162 ^ 




171 


0 


162 


13 


175^ 


0 .j>. 


^ 162 


fl2 


,174. 


0 


* ; 163 ' 


. 8 


171 


0 , 


163 


. 12 


"175 


0 


163 


': ■ 9 


- 1 72 



Xi^U PERCENT 

JIME DEVIATION . 

(SEC) FRCK CPTI>UM 

-60^3 ' 0.0 

58.7 0.0 

65.7 0.0 - 

114.0 0.0 ^ 

96.9 0.0 

100.8 0.0 

13^6.2 ; . o;o 

118.0 0.0 ■ 

166.1 0.0 

59.1 ^ 0.0 

. 58.6 "0#0 + 

56.6 -0^0 

52.3 , 0.0 

92.6 0.0 * . 

107.7 0^0. 

90.2 ' 0.0 . J, 

125.2 0.0 
125.1- 0.0 , . 
119.2 0.0 



NUMBER OF ROWS 



DENSITY 

;orb4'. 
:'*^o.o4 

0.08 . 
0.08 
0.04 
0.04 

o 

0 

0.08 
0.08 



100 



— REDUCED^ 
COST 
PERTURE. 



0.1 jS? 

r.^0^ 
0.1 
1.0 
0.1. 
1.0 
1.0 
0.1 

0.1 
1.0 



^WfHR^OF^OtUMNS . .NLMBER^OF UpPER *BCUNOS 



CPT. 

SOLN, 
TYPE 

O ' 
-^0 

0 

0 

2 

2 

2 
. 2 

2 

2 



200" 



' IJ^ATIONS CPU 
PHA;SE 1 PHASE 2 TOTAL V^'«HC > 




436 
436 
468 
468 
361 
361 
361 
397 
397 
397 



142 
^3C 
156 
178 
64 
27 
65 
7C 
91 
85 



578 
566. 
624 
646 
425- 
388 
426 
467 
488 
4 82 



22s. 4 
2ir.7, 

468.3 

471.7 
: 165.6 

152.2. 

162.1^, 
.363.1 
* 382.1 

377,4 



PERCENT 
DEVIATION 



FRCM/CPTIMUM 



0.0 

0^0 

0.0 

CO 

Q.05 

1 .8S 

0.48 

0.08 

0.0 

0.0 



V 



NUMGER OF RTOS NUMBER QF COUtMNS NCMBER OF WPPER BOUNDS 





too 






REouceo 


OPT * 




COST 


S0UN# 


DeNSITY 


PERTune* 


T YP 6 


0*04 


0.1 


0:- 


0*04 


0 • I 


V 


o;o4 




0 • 


o«Od 


0^1 


0 . . 


0*C8 


1 #c . 


a 


€•08 


l.C 


0 


0.04' 


0.1 


2 


.0*04 


0.1 


2 


0.04. ' 


1*0 s' 


2 ' 


o.oa^ 




.2, ■ 


o.od 


l.C 


2 . 


0»06^ 


i*c. - 


.. 2r ' 









200 



ITERATIONS*. 



PHASE 1 PHASE 2 TOTAL 



385 


,204 


569 


223.5 


385 


..-9 


• 394- 


145.8 


385 


162 


567 


225.8 


421 


. 143 


564 - 


.403*4. 


421 . 


■ 63 : ■ 


484— 


^349.7 


421 


. 41., 


462 


323.5" 


285 




356 ■ 


154*6 ^ 


285 - 


12.9^ 


414 


176.0 


285 


133 . 


. ^18 - 


169V5 


382 


.150 


532 


376.6 


382 


• . 86 . 


468 


33U6 


382 


rei 


563 


420.0 



50 



CPU^ . PERCENT , 
TIME OEVXATICN ' 
<SEC) FROM OPTIMUM 



0.0 

0.52 

0.0 

a.o 

1.02 
l.,6C 
0.09 
0.0 
-CO 
0.0 
0.93 
0.0 



• y 



NUmIbER^ COLUMNS NLMBER OF UPf?ER , BOUNDS ^ 



lao 



300 



REOUCEC 
. COST 
DENSITY PERTUPe« 



OPT. 



ITERATIONS 



0;04 
O.04 
0.04 
0.C4 
0.04 
0.04. 



0. 1 

1. C 
1.0 
0.1 
0.1 
1.0 



SOLN. 
TYPE 


PHAsfss^; 


PHASE 2 


TOTAL 


0 


572 


< 42i 


995 


0 


606 


171 


777 


0 


573 


2SC . 


623 


2 


473 


ICS 


578 


2 


473 


108 


581 


2 


473 


248 


721 











CPU PERCENT -5 

TIME DEVIATION / 

(SEC) FROM OPTIMUM 

402.1 0^0 
340.3 2.90 ^ 
342.5 1*41 ^ 

292.2 0.73 
294.0 0.7O -H 
305.5 0.0 



NUMBER OF ROWS 



100 



NUMEER OF* COLUMNS 
300 * , 



NUMBER OF UPPER BOUNDS 
50 



REDUCED. 
. COST . 
DENSITY PERTURB« 



OPT. 

SOLN. 

TYPE 



ITERATIONS ; 



0.34 
0.04 . 
0.04 
0.C4 
O.04' 



0.1 
0.1 

.1.0 
O.l 
' £.0 



PHASE, 1 

— r 



0 

6 
2 

^2 



ERIC 



PHASE 2 -TOTAL 









579 


153-^ 


732' 


579 


88 


667 


579 


41* 


994 


376 


295 


671 


376 


. 228. V 


6 04. 


} 







CPU PERCENT. 

TIME : OEVIATICN 

(SECJ FROM OPTIMUM 

2ea.2 . 0.20 

270.5 ' 0.32 -H 

356.9 0.0 ^ 

308*. 2 0.0 

^62.8 . 0.0 



-Iff • NCM8£« CF UPPER BOUNDS 



■ V 



-3? 



AEOUCEC CPT. 
COST. SOL'n. 
DENSITY • PEMtUne* TYPE 



C#04 
0#04 
0«C4 
€•04 
Cl«04 
0«C4- 



^ ^ITERATIONS 
PHASE I PH ASE^a'^ 



0.1 

- i.c 



.0 

0 ' 
0 

0 V 

0 



1663. 

I77'4 
927 
954 
947 
^79 



•32 
6C 
31. 

a 

68 
12 



CPU • PERCENT 

TINE , DEVIATION 

TOTAL (SEC) 'FROM - GPTII^UM 

1695 1259.2 . 0.0. ^ 

ie34 J322.9 - 0.0 : 

/.'QSe 869.2 * 0.53 ' 

962^ \8«/.7 " 0.52^. + 

1015 927^9 3»74 

992 . 925*2 6.81 ^ 



•1 



NUMBER OF iSCWS 



NUMBER -pF COLUMNS NUMBER OF UPPER 6CUN0S 



155 



OBtEOOCeC - OPT . ■ 
• COST . SCLN. 
DE^ITY PERTURB. TYPE 



0-.C4 
0^04 
0.C4 
0.O4r^ 

o;o4 



O.I 

1\0 

^ I.c . 



^ ' 0^ 

^ 0* 



30O 



SO 



ITERATIONS > CPU PERCENT ^ 

. TIME OEVIATtCN * 

PHASE 1 PHASE 2 TOTAL C^C^J FROM OPTIMUM 

-• .1430 88 • 1518 n 1587.? ^ oToi*^"" 

7 ,1393- 6 1399;^ 1455.7 1.C5 

.. 787 : 198 ^ 985>: 1187.4 0.07 ■ 

. 787 - ICS ^ 892 1054.6 . 0.0 >f 

•787 -91 - 878 10C6.7 1.79 > 
• ■ ' . -.A ^ * 



. ^ ■" for Che optimal 'solutioa type 

/ P*'; ^Indtcices degeaerate 

• ^ ' 1 .• . iadicates unique ' . 

•.'2" " indicates alccmative 




^SESSmvriY ANALSTSIS'-FOR PARSCTOIC no^ilinear 

pimwimj:;- using penalty meihcds 

■ r. " - ^ 

^-^ * / 

. Jtobert I/. -AiitacxDstv' . 
• ■ U.S..05ast Guard Headquarters" 

• ■ ■ . ^ '< . ' ' ' ' 

Anthony, V* RLacco " - ■ j- 
D^aartniait of Operations Kesearch 
^ Uie George V3&shington Oiiversity 



Abstract 



* ' Recently, it has been" shown that a class of 
penalty function algorithms can readily be adapt- 

, ed" to^^ener?tte - sensitivity '.analysis information 
for a large^ class of parametric nonlinear pro- 
granning. pn}bLLfims.' particular" estimates of ^ . 

■ the ^.partial -derivatives., twithre^^ to the prob- 

. lem parameters) of the ccnponents of a .solution 
•vector and the optiml value function have been 
s u ccess f ully calcalated for a nunber of 'nont±ivial 
.examples. -^"IDae^a^jprpach has been^ijrplemented using 

. .the well-Jcnowh- Sequential Orioonstrained ilinimiza-- 
tion fe chn ique (SCMT) ccnpjtser program. Biis. . 
.p^)er briefly surnnarizes these resultjs, -presents 
additions to the ocnpiter progr cg n that include 
a screening -device for. eliiianating calculations >- 
-assg c iated r with less igportant parameters/ and ^ 
illustrates 'the ki^i^f iitf ormation ^iSfe can be* 
generated by applying the technique a well- 
kncwn inventory model. 



1*. - IntxTpduction - ■ . ^ - - 

^ Initial numerical results resulting fran. the' ' 
i m p] e m entaj-ioD jof a "penalty function technique , 
"l/'tfar obtaining sensitivity inf omation in parame^ 
tz^j D o n li n ea r progfannong ,were givm by Annaoost 
.{1974)-.- ohe vjork^is ±>ased on the^ ' 



and 



theory develc^)ed by Fiacco and Mo^ccnnick^ (1968) 
and esctended by Fiacco (1973) . This p^ser repbirts^ 
On refinements and extensions of the oqpputatic^ 
al procedures ,inplemerjted' ly Aimacost and,ffylariaer 
{1973J and Armacjost fi976) , using the SCMV • 
. %rsiqn 4 dcnputer oodfe vath the iogarithnac-. ^. 
. quadratic loss .penal-ty.' furiction to estimate- the' 
partial -derivaLtives of the soliitian: point and the. 



valxae-/ the denvaldves-" 
re^ject -to ,the specified - 




objective function 
here lieihg taken 
prdblan parameter; 

Fiacop (1973) ^deJveloped 1±e necessary general 

. fonmtes for the partial derivatives of tte *'opti- 

. .mal vaiue"fuixrtion/'/the.cQrf|x^^ 
: r;sol\2tion point aiid . its associated optimal 'Lagrange. " •- 

' mnl ti p i . ie r s . for a; large class' of- parametric non- 
. W nmr prDgxaiuiiing problems corpos^d" of twice dif-. 
•^•'.^ferentiah^ 

'nation, formulas iri terms "of the -vjell-known^ ,r ■ 
^-^y irithmic-guadratic penalty function.. Recently', - 
tryLioost and;-Fiacco -(1975) particularized aiid 
^' ™^™^i ' i>lifie a / these f6r m jlas fiar various problemrgv - . ♦ 

^ structures-and develooed fonnulas for the first! . ✓ . 



and second derivatives of the <2ptimal value func- ' 
tion of the given problem. Mditicnally/ Arroaoost 
and Piacco (1976) have applied the general theory 
to easily prove t±e\vjell-known result that/ 
the parameters, axe tiie. riciit-hand.side ccnponents.- 
•of tSie con stra ints, the" qptiinal jLagrange multi^ 

, pliers give the gradient of .the. optimal ^^ue . ■ r 
function (with respect to the pararr^ters) . - Further, 
it TOS shown that the first, derivatives of the 
Lagrange multipliers give the ooqpbnents of the 
■ Hes si an . of the optimal value function," and escplic-; 
it formulas 'were developed for the Hessian, in 

' 'te33rs of the problem: functions - 

'. In their first report on ccrputational eiqjer- 
ience, Armap^st. and Fiacoo (1974) concentrated . . 
•primarily on presenting, ccqputational ea^efience 
associated with the calculation of the f irst/der^ 
ivatives of .a local solution point. practical 
' implemen t ab i 1 i ty^of the appi ' o ach Jwas d emo n s tra ted. 

In a subsequent p^ier, Armacbst (1976) re- " 
ported^on 'additional ocnputational' experience, ^. ^ 
' .focusing. 9^' the calculation of: the derivatives of 
the cptiital value- function, and ■ the Lagrange' muiti- 
• "pliers,- also iirplementing *a' potentially yaliaalale - 
refinement that'-allcws for ccn^terizeil screening- 

■ rfor .?kqr" pa ram eters. 'This" paper nay be .regarded * 

■ as a cantinuation and anplification of the Arma-r 

^ For.pj^bblems involving a large ntinber of :* 
parameters, a very large number of ^partial derl- ' 
-vatives may be^'calculated if one proceeds indis- 
" crimii^tely> ■ This is not only tirner-oonsuming,, . 
tet 'may eO^/te^ 

must evaliiate' the oveSKLLI" significance, of .the J 
I results l >Cde measure of the .Icteter- is the effect 
I; of* a perturiatibn on the s61u€iDn value . It is 
•^quite-possible and often deserved in practice ^^^^ 
"that the cptinal objective functicn value is much 
more sensi-tive to a. few of the'iiHny parameters 
pres^t^ With , this in mind, tte' method ^developed" " 
■by Arfnacost and Fiacco (1975) to^estimate the 
first order- seiisitivity of the -Optimal value func- 
- tp.on -was incorporated yi the coiputer- p rogr ai n to- ' 
provide an cptiori' fbr prelimitiaty screening of the " 
parameters to eliminate further calculations inr ' 
volying po^urbations of pa ra meters, having "little"' . 
'. effect; on the qptimal .valxae funct^ (A .-user .^^ 

can 'easily irrbxciuce^his^ of ■ signif i-; > ; 

' . cance, m this ^determination. ) Using> the. fbrmalas 
developed by Fiacco' (1973) , a second option is 
includ e d which permits^ the calculation of tJ|e' , - " 



J 



In Section 3, a sensitivity analysis is ogq- 
dnrtea foira mlti-item inventory modfial developed 

Sctaady aixi Choc (1971>. for the U.S. wavy. 
cc aip l e aneilyzed is the sazne snail one treated ^ 
Schrady and Choe, thouc^ readily extended to a 
large-scaOe TirrteT> Ohe results illustrate the po- 
tential value of a. detailed 'autooetted sensitivity 
^azialysis in prectical situations, and hopefully 
dramfttize the nuinerbus ricfa interpretations and 
i n si g^ that can 3be derived fron this informa- 
tion, as veil as.indicating the caution that nust 
he taken in mking valid inferences, 

- The recently obtained basic theoretical re- 
sults validating the oociutatiGnal algorxthm ^are ^. 
sonnBrized rather coopletely in the next section ' 
\» tiiat the paptt ini?^ 

.2- S uj4*J£t inq flaecgy 

^ ^ ^he paranetric mathegaocal prograznning prcd>- 
,leDas co n sidered here are of the fom^ * 

imnindzfe^f (x^c) 
sid5ject to g^(x,c) > 0 , i«l,-..,m , p(g) 



itere X is the usuad vector of variables and e 
is.a-k-ccnijonent vector" of rajmbers called ."parairt- 
ctcrs.^ Jt is desired xiltiinately to develop a 
ccBpplete cfaaracterizatioai of a solutjion - x(e) of 
^?3Ddilan P.(c) aiis. a function of g . * In our cur^ . 
\ rent work, we have ocnqentrated on certain re- . 
^oerrtiy oon|3utationally " tractable iiea^^ S 
' -'change in a sbliition as g is*''perturbed from a' . 
' ^jecified value. (Without loss of generality, we . 
assune that, the gaecified value is e = 0 *) ^ 

'Wbenx«rtaih->a^5aiptiQns are satisfied. 



iliSCOoTlSTS) and JVn^Krpst and Fiacco (1975) have 
characterized the ^„*first. order ^^ensitivil^" of a 
^'Ti:^t^^'^^ixker Triple" andxi^he-^irst and seccaia 
order "^sensitivity of the optimal value function 
of Problem .P(d) ♦ ^ (^a«se quantities arfe defined 
as the .theory is presented.) Aiditionally, they 
have deweloped forndas for efficiently estimating 
this sensitivity /when the logaritirtic-guadratic 
loss ^penalty function algcaithm is used-to solve 
Pr obl e m P(e). The roain theoretical .resialts are 
saanarized here. ] j ' v 

"ae lagrangian for Ptpblein*P(e); is defined as 



L(X,u,w;g)- h f (x,c) 



- ^ u.g. (x,e) 

i^l ... 

+ I w.h. (x,.g) 

1=1 ^?.}. 



i«l,.*.,m and Wj , j=l^,v.^p arei 



J " ;uhi^ 

; . TlagMnge in iltipli e rs" associated \ath the. inequal-. 
• : ity and equality constraints, re^aectively. - Any ' 

• .. 'wctac^ (x,u,w). satjLSfying the usuaX (first or^ter) 
Kuhn-^lSjcfea: cotiditic^ ifiacoo and Ititarini^^ 1968) 

' ^ Prphlaa p(e) is, called a>JCc±n-Tucker/t^ ; ■■ 

•CD! r - The^^fcllowiJW' fbiir assumptions aire suff icidit ' 
Lii^ establish tbe desired results and are assumed 
; _to hoJii througjfiut the pap^ 



Al" — n» funqtionS defining Problem P^c)" are 
twice conti n uously differen tiahl e in 
(x,e) in. a •neighborlTOod of (x*>0) . 

A2 ae second ooctier ' sufficient conditions for 
; • V a loccil miniTTMTubf Problem P(0) hold at ' 
X* with associated ^Lagrange nultipliers 
• u* and w* • ' ' ' \ 

A3 — Bie gradients ^V^g^(x*,0) (i.e., - 

/ . (Sg^^ (X* ,0)/3xj_> . . . , 3g^ (x*/0)/3:;^) ^ ; ^ 

the- superscript * T denoting Sansportationj 
for all i such t]bat gi(X*^0) = 0 , and 
7 h. (x*,p) , j=l,.;..,p are, linearly, inde- 

* pfendeiA. ,^ _ 

A4 — Strict ootrpleniEntary slackness holds at 

Cx*,0) (i.e., ur>^0 for all £ such that 

- g.(x*,0) = 0 - 

'gieorgn I r (local characterization of a Kuhn- 
Tudcer Tnjjle (Fiaooo, 1973), of Problem P(e).) If 
assunptions Al, A2, A3 and'A4 hold for Problem ■ 
' P(e) at .(x*,0) , then 

(a) X* is a local isolated minimizing point of 
Problem P(0) and the associated Lagrangei , 
multipliers u* and ^w* aretmigue;. 

(b) for e.7 in a neigfiborhood of 0 , there 
exists a unique, once continuously differr • 

^ . entiable vector function V i 

y(e) = (x<€),u(e) ,w{e)) satis^ang the . 
second order sufficient, conditions for a- * 
local miniimiu of Problem P(e) such that. 

y{0) = (x*,u*,w*)^ = 'y* and hence, x(c) 
is a locally iiniguer local minim jn/of Prob- 
■ lem P{e) vdth associated unigue Lagrange 
. multipliers- and w(c)' ; and 

(c) . 'for g' near* 0 , the set of binding' ineguai-^ 

ities ds unchanged, strict- oomplanentary 
slackness holds for for i such 

. that g:(x(G},eI = 0 , and .the binding con-! 

straint gradients .are linearly independent 
. at x(e) . 

mus result provides a characterization of a 
. locals/solution, of Prc^ilem P(c) and its associated 
optirBl Lagrange multipliers near g' = 0 . It 
generalizes;^ a theorem .first, presented by Fiacpo 
and rfcCc3(nnick- (1968, Theorem 6) and is closely ■ 
. related ^ a generalization of the same' theorem 
provided 'independently by Robinson (1974), It 
shoMS that the Kuhn-rGcker triple y(e) is unique, 
and weU bdbaved, uix3er .the given oondi 
Sinc e y(G} -;is once< differentiahle^ the partial 
- derivatives of the.ocnponents of . y(G) are well 
. - ' d ef i n ed. Otis; fact and Assunption Al aiso mean 
that the functions defining Problem^p(G) are once 
continuously differentiahle fimctions of e along . ' 
the "solirtdoh trajectory" x(e)/ near; e = 0 / ' : 
. and the lagrangiari is a- once continijously differ*- '; 
. entiable function of . c along the TCuhn-TuK±er ' - " 
' joi^ trajectory." ^ . ■ • 

y, ' Wfe are thus- motivated to determine a means to . • . 
■calculate the various partial derivatives^ siiice . • 
'.^^tbife yields ^fij^ order estimate of the locally 



pptinal Kuhn-Tocker triple and the problem func- 
tions near g » 0 . j ' ^ 

^ JDenoteby 'V|.x(e) ^ (W(e)/3Ej) /i=l,...,n/. 

, the-, n X k matrix of partial* deriva- 
tives of . x(g) . vdth respect to c , and define " 
v^uCg) . and 7^w(g) in similar fashion. T^fe 

then define 7^y(c) (7^x(e) ,7^(g) >^<g) )T ; an . 
(n+irHp)-x k matrix. ' . . . 

•^^ihen y(c).' is available, 7'^^(g) c^ be cal- 
culated by noting that Occlusion (b) Qf the - 
. theorem inplies ' the satisfaction- of the Kuhn- 
Ti3cker conditions for PCe) at y(e) near 

G = 0 , i.e.,-: 

, u^(e)g^[x(e) ,e] = 0 , i=l,,..,m,. (1) , , 



(2) 

(3) 
(4) 



j=l, . . . ,p) and such that " 

x(0,f) xW^O) = X* ; . 

. m • . . . / . : ^ . 
liin^ r 2 Jin g. [x(b,r)] ='o 
r^ • >1 - ^ 



lim- (l/2r) 
r-O 



Z h.[xW,r),0] =0-;:-and- 



lim VJ[x(0.r),0/r] 'f.(x*,0) 



Tlie fcllowing theorem extends these results for 
Problem P(g)7 vdiere e is allowed -to vary in a' 
• neighborhood of i'O 'V and provides a basis -for 
^jproximating^ihe sensitivity infozmation associ-r. 

ated with Problem P ( g) . The_notation i—V^ 



The_nptation-_7 w derxzites— 
• X 



hj[x(G),£] = 0 , 



o=i,.- 



Since'Hiie Jacobian -•H(g) - of this systan with . 
respect to- (x,u;w) (i.e., the matrix obtained 
'^by differentiating the* left side b£ (1). with res- 
.pect^to the oonponents of (x,u,w) )- is hon-\ 
singular under- the cp.ven assunptions, tte total 
derivative of the system with* respect to g is 
well' defined and must equal zero.-. UrLs yields 

■ . MCe)v^(G) = N(g) ' 

where, N{g> is the negative of the Jacobian of 
the Kuhn-OXicker- system with re^^ect to. g , and 
hence ' - ' *-v ■ " 

^ * . - V^Xc) =^ M(g)- -Sj(g) - ' ■ ■ ■ 

,The class of algorithms^ based on- thdc^ oon- - 
tin uous ly dif ferentiable penalty, functions - can be 
• used ^fethout additional eissunpticns and without 
requiring y<£) to.provide an estjjnate of 
V' V^yCG) . Furthermore, most 6± the information 

required to make the, estimate, is already ava-ri- 

in thev typical irtplenientations of these ' 
al5pnttimfi</ Here, we use thie logarithmic- - 

atic penalty functim for:? Problem P-(g) ^ ' , 
(Piacco and McCtoick, 1968} defined- as : " 

^ . " m ■■• \ ■ ^ ' ' 
W(x^G,r) = f (x,g)' - j: S in g. (x,^) ^ 
- ■ 1=1^ ., -' 



the matrix of secord partial d^ivatives of ' W 
with respect to x ' . . 

. riJieorem 2 ; (Relationship tff ^^^ions .of 
ftDoblen P(e) and minima of 96c/G,^^^ (Fiacco 
1973). . If Assirtptions M "/A4 hoS^then in a - 
neighborhood about - {e,r) ^(ffere exists a ' 

^'unique once continuously difcferentiame vector 

function y(G,r) = [x(G,r) ,V(c,r) ,w(t:,r)]^ satis- ' 

- fjdng,- , ■ i ■ — ■ 

■ / u^g^^Cx^G) =-f i=l,...,m ,^ 
".^^(^'^^ ^'.Wjr j=l,...^p , 

with^ y(OrO) = (x*,u*:,w*) and such that, for ai^ 
{G,r> vnear (0,0) and r > 0 , x(G,r)' is a 
locally unique unconstrained local minimizing" point 
of -W(x,G,2;) g^[x(e>r),G]. > 0 , i=i^..v,mv; ^and• 
^^J[x(G,r) ,e,rl is positive definite-' ; \ 

i ■ , ' .-s . - ' 

Qorollgry 2.1 : - (Convergence of estimateSi-^^jising 
.W(x,G,r>.., (Fiacco/ 1973) .)" If Assumptions Al, A2, 
7^ and A4 hold for Prdblem P(e) , then for any ^ - 
near 0 > * 



P 2 * 

+'-(l/2r) Z hT(xyG) 



(2) 



Uhder the given assunptioris, 'the follc^ng 
facts are Joxwn for Problem P{0) from "penally 
function theory ^(Fiacco and McOormick, -19i5S, ■ 
theorems 10 and 17): ■ , 

(1) For r > 0 and sma^l, there exists a .^^ 
' .unique once continuously dif ferentiable* 
^ vector "fttnctoOT xfO,r> such that x.(0,r). 
^ ^ locally xmique minimizing' point of ■ 

- W(x,0,^i3 . in"~.R*(0)' = .{x:' g, (x,0)" 

0 , is?l,...,m , and h. (x,0> = 0 , ' 



(a) 



(b). 



y(e,r). =^ y(e,0)-.=,y(G) > the Kuhn- •- 
r-tOt , ■ . . • . 

Tub k e r txiple ^ch^ractearized in Theoran 1; 

and . * '/ ' 

lim ' V v(e;r) = v(g^O) = V y(e) ! ' 
r-K)"^ r . " , e 



This result motivates' use of V^Cs/r) to 

estimate. -V^fc) , whfen g is. near 0- and r 

- is- near' 0 , once y(e,r) is available, iheoren 2 
- provi<fesj^ie basis for ah ef f icient'^caldilation of 
^^l^,r) . 'Since, -at a local /solution point - 

x(c,r) of. ?7(x,£;r) , it follows that ' ' ' 



.7^I:T[x(e,r),G,r] = 0 , 



(3) 



we; can differentiate (3) with respect t® G^to^ 
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(4) 



+2 W.7 h.(x,e) 



an 



• '-^ By 111600:^ 2, is positive definite for 

(c^rX ^ near (0;0) .;^nd r , so 7^ jjais 
inverse and V^xtc^r) ='.-i7^-7(x{e,f) . ; 

^J^[x(c,r) ,c,r] \" - ^ ' ; , V 

Also/ siiice' ' ^ . - .'^ 

u/tc/r) = r/g[j^(x<cVr)ve) , -i=l,..,,in / . '(5) 
"and 

w.(e,r) =,h.(x(c,r),c)/r , j=l,...;p, (6) 



(c) v;f*(e) = 7 (7'£.-{x(: 



' for U;r) near (0,6^J and r > p , the^ equa- 
. •tions^'can be differentiated with^i^re^ject to e 
•to ofcftidn 

• 7^x(c,r) + 3g^(x(c,r) ,c)/3€] • ^ (7^ 
-^cV-'*^^ " (1/r) [v^h/(x(G,rJ,e) - - 
.^.v^x(G,r) + 3hj{x{G,r)^G)/3c] . ^^(8) 

.Solving (4) and calculating (7) and (8) then 
yields the , cccponents of 7^(G,r) , v^ch can be 

losed to estimate 7 vfe) for (G^r) near 

, Olie'next^ results .exterjd this" thebiy. to an ' ' ' 
-analysis of the qptiinal Vcilue function of Problem 
PCg) along the Kbhr^Tocker point trajectory 

[x(0/U(g),w(g)]'^^ . • V 

- Cie optimal value function is defined as: 

f*(e} = f[x(€),e] , ' 



,.lhe logarithmic-quadratic loss penalty function 
(2) can also be used to provide e<5tiinates of the 
first and second order sensitivity of the optimal 
value function- Let. the' optimal value penalty func- 
tion be defines as W*(c,r) = w(x(e,r) ,G,r) . 

. ' ifeorem 4 : (First-and second order sensitivity 
of ?J*(Grr3 and estimates for f*(e)^ViVrroaoostx 
and Fiacoc5\ (1975y .5- If Assu^rptions AL- A4 hold- ' 
foTL^Ezobleoi^EXEO^, thCT fnr.--(-e-yr)— near^^OyOy — and- 



r > 0 W*(G^r) is-a .twi^ oDnf.imo.isTy different . 
tiable function Qf ''ca.and' ' . /* ^ • 

^ (a) lijn: W*Ce/r) = 'L*(g) = f*(e) ; ... ' 

r^+ ' ' 

■r ... ■ - ■ . '■ ■ ' 

(b) V^V7*(G,r) = 7^WJ7^X + v^w = 7 X(x,u,w,c), .: 



(X/U,w) = Cx(Q,r)*,uCG,r) ,w(Grr) ) ; - 



(c) liiri 7 V7*(G,r) 
r->0 ^ 



='7r(x{G) ,u(e),w.(G)^G): 



- • = 7 f*(G). ;. 



(U);; 



v\j^U,r) = 7^f*(G) . 



and the "optiiral value Lagrangian" is defipei^asr 



I.*(g) = L[x(e),u{G),w(G),G] 



(10) 



• Tfaesorem 3 : - (First and second order chang^ in"'; 
the o nfenal TOlue function, Annaaost and Piacco - 
(1975)^- -If ^assunpti<&s Al.- A4 hold for ifeob-;^ 
lem P(e)/ then fipr g near 0- , f*<G)- is ^ 'r^-:^ 
twic^ continuously differentiahle function of /^^^ 
and.::- •:■ .... : " . ^ -^^^^ ' - 




-result provides a - justification for 



bp: 



(a) f*(€) = L*(g) ; 



(b) 7 ;f*(G) = 7 L(x,u,w,g) 



(x,u,v^) = (x(e) ,uCg) ,w(£>} 



m . 



,, ERIC 



* .."/ 



• \ 1=1 ^f-^ . 



f * ( g) , 7^f * (£) - _ and ' 7^f *Xs) ^ . . 
W*^e.,f) 7^W*(G,r) and " 7^;* ( G,r) , respectively, 
r is pQsitive . and small enou^. * ■ 

Since; corollary 2,1* and continue 
"■lim f .{x(G,r) ;e) ^. f*(c) / another estinete of ttie v 

■r^ • ■ ;.- ' . "■ ' . ^' . ".. ■ ' ■ 

.**^V*^' optiinal value function (9) iS' provided' by * - • 

- ' " V f ^^Ti") = f CxCc^rKe) / when r- >, 0 -and small. : . ^ , 
- . V , Qatrect l^licatiqn of the- chain rule -for ' differen-''- 

, \ " V tiationj then yieTds, f oar- X - x(e,r) / \ 

\ - ■ \ ' " '■' . 

7J5*(c,f)>>^f (xrt)7:x(e,r)> 7 'f(^^ 

Under^the gxven assunptijons^ 'contimaity. aJso assures . 
* J -ttiat 7^f#(e,r) V f^T^J^-as r ^o"*^ ."^ ThusJboth ' 

G^ ■ •■ . - G' - . - ■ * 




. V^f 7^2:5 . and 



Co * 



^W*.(c,r) oar/ estimates of 



":fcr- r * sufficiehtly snail. 



• - it should be" noted that these estiirates are 
functionaLLly related since " ' 

- 2: u. (V g.7 x(G,r) + 7^g.J 



J. w.(7 h.^ x(G,r) 
j=l J ^ 3 



v^h.) 



. Ptom this e:q>ression, it is^ clear- that" 



7^f"(c,r) 



is the better estimate of v'f*(e) ^ tiae \rCTHtining 

terms "in 7^W*{c/r) siirply cxnstituting "noise" 

that is eliminated as r o"*" . However,, by using 
the expression for 7^w*(c,r) given by (U) , 

_-^^W*(e,r) can be evaliaated without necessitating 

. -1±e. calculation of 7 x(e,r) , vAiich. is required 

■ to cqppute (12) . . Thus^ 'the cruder but coiputation- 
ally raoch c±keaper estimate of ' 7ef*(c). given hy 
Equation (11) }ias now b6en introduced as an opticn. . 
in.- the ouLputer poro^^am as a preliminary screening 
' .device to. identify crucial parameters* l?estric- 
tion o'f svibseguent ^c^culaticans to' these paraire- . . 
-fifers,. and'.other cgtlculations such as the sharper 
; ' .estinate^of ■ 7f*(e)\ given by C12) are provided asv 
ac3ditional- options-'^ ,^ ; 

In sumtary^ the basis for 'the estimation pror*^ > 
utili2ed,.here for a ^)ecif ib protaeii; say 

■ ;£OTOlent P,(4ErO/r is' the ^^ninimizatLon of l±fe penalty 

function -WCx,;e;f) given by (2)'_ This yields a 
-pomt x(e^r).. whidi msrtr be viewed as' an estimate" 
Of a (local; solution x(e) of Problan P(e). The 
-estimate f {x{E,r) ,c) of f*(e) is iimfidiately " " -■ 
available x{c,x) has been determine. iiU . 

. associated .optimal Xagrange railtipliers • u(e) " arid 
' y(c) are estimated by using' the relationships, 
^gxven^in (5) and (61 . -respectively, 

■ ^ ' if .desired^ all of the first panftal deriva- ^ 
^txves of >(e,>) - (x{c,r) ,u{e.tj ,w(rr}T ■ 
^^^spect to E , ^ an estimate of: v^y/e) . my be - 
-cbtained by first solving • (4). and d5en mplying 
; (7) and (8).. 'If .-t^ie full-matrijc / xCc>r) <is ' 

calculated^ .thq;i '7rf*{ty is est^ted ±y (12)\ ' 

However^ if il) is desired to. eliminate calcula- ' 
. -.tions involving :parameter5 having le^ effect on 

local changes .of f *'(?i) / t^ screening device ■■ 
.V described aboTO is used.' Uiis entails initial es- 
timation- of : 7^f*(e) Jby {,11^. Ccii?X3nen>s.of - e . 

OQiScesponding to the respective ccxipbnents of ^ * 
,7^f*t'c) that are- deemed inconsequential/ m^ then 

be deleted and wuld Jtot totef< into* any ^t&seguent 
calculaticcs... In parti-cular^ it is en^iasized that 
(xj^u*>^) / f*«(E) and , 7^f*{c).. • Inay be estimated^ 

^; jr(Gfr) f (x(t:,r):/e) • and (yb) / respectdv^y/ 



without calculating ai:y conponents of ^^'(c,r) , ' 
once' x(e,r) is 'taown. - , 

3, Exairpie: A Large-scale !lalti-_item 
• . , - ■ Inventory ftodel 

traditionally, inventory models have been formu- 
•;:lated to minimize some function of the ordering^ 
, holding "anS shortage "(or backorder) costs 'sti^ject 
to various' oonstraijits." Schrady and Oioe (1971)/-. - 
have forxnulated an iiiv^tory nodel vtech appe2u:s- to • 
have much greater relevance , for- an inventory system 
in- a noncaiiiercial eriWronrnent/ such as institution- 
. al /orMnilitaly^ • Olie costs used- in the traditior^ 
rnodels he quite airtificial and the real ofcijec- 
• tive of the system is of tea naximi^ation of ai mea- 
. sxare of readiness pr^^^ervice, here assumed to be? * 
' ^equivalent to mniinization of stockouts.. *Tn addi- • ' 
tion, the stock points of such supply systems are 
inevitably constrained by inves.tment and^reorder - • 
/workload liirdtatidoiis^ ^ _ .; - . ^ * • 

■Sciirady and;aTpe"*5 multi-item inventory system- . 
assumes these constraints *^ong with the specific . ' 
objective ot minimizing the total time-weighted ' 
shortages- The decision variables'-are .taJcen to be 
tJie "reDrdeifquahtitdes"' and^t^ "reorder points," 

■ respectively, hoi^iiiuch to order and ^whrn to order 
each item in "the inventory* ^ three-item exanple 
problem was solved by Schrady and Choe (1971) using 
the SUMT qorrputer' code, (Mylander/ et al., 1971). " 
Subsequently, .Mccormick {197Z) ^showed. how the spe- 
cial structure of this inventory moc3el" can be used ^ 
to facilitate^ the use of- the SIM" code to solve 

. very large inventory prctelems. - He also extended . 
the model to include constraints on storage volume 
and the probability of depletion of critical' items.; 

■The model 'and exanple presented here are the • 
original ones'due' to Schrady and Choe. Ihe penalty ■ 
ftiyrtion technique described in the preceding sec- ■ y. 
tion was xised . to solve the example and cailculate 
the parti^ derivatives of 'varidus quantities of 
interest, with, respect to e^c^ parameter involved 
in. fiefining the model . (Tlie analysis . can ■ be applied 

to the- extended model vdthout difficulty. ) 
■ ■ ' ^ .. - • ■ ' ■ ■ . ■ 

•Detailed development/of the rrbdel is beyond the 
scope of this .paper, ' Bie :interested reader is re- 
■ferr^ -to the Schrady-Choe and rScOormick-papers v 
Here, Wje^ give a suirtnary .treatment of 'the various . 
cOTc^^tior^ and relationships lapon \^ch the ' model ^ 
is based I'fe then tabulate the results obtained 
in solving' the resulting nonlinear progranming prob-. 
lem and applying the sensitivity analysis method- .. . . 
plogy- A nirrber of obserVaticsis and interpreta- 
tions; are . offered to" illus'trate the many, uses - to 
v*iich the sensitivity information might be applied: 

. It is as^rned' that the amount of each item in . ^ ^. 
inventory. .is alvays known, that all demand which- 
occurs when the-, on-hand stock is zero is back- ■ 
ordered,' and ^ that the'^demand vihich occurs during • " 
the tine betsieen .^the placement pf-an"Ca:x3er'.ahd its 
receipt by the stock point, (i.ei , - ttie "lea4, tiir^- 
demaixi") is hormally distributed' with- known mean 

u. and variance c. ^; • ■ , . - . . . 



For .the item,: let 



c^. = Item unit - cost 



(in dollars) 



'X^-"=='*mean demana per unj.t tiine (in' units) , 
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■•.•V ■ ■ ' V- •.. • . ./ ■-■ 

.".r. = reorder point/ , • ' - ■ 

T, ■ ' (x) .= the Normal; iO, 'density* functibn^ 

o(z);=^Ct>(x)dx = the-Ji^^ (O^i) corpleinentary 
c lati^ distributiOT;;f^^ 

\r ' In additiiiri, let' K^ be the investment, limit 
• -in doliarSit K2 the ^onber of orders per xmit of 



time. that constitutes 
-H the total nuntoer of 




war3cload.. limit/ and 
in t±ie invenfcry. . 



. It can be shown that the ^-expected -tiire-wei^ted 
shortage of item i at any . point in tisca is given .by 



where 



mie e^qsected cn-hand inventory of item i is given ; 
.. by +-0^/2 - + ^i^^'^i^ ^ ^ expects? 
nuntjer of orders placed per- unit time for-item i 

-.is x./Q. . ; . ; - 

Iteing the above e:qpressioins and assunptions/ 
Schracfy and. (Snoe (1971) . indicate that meaningful . 
. : app ro^giiiations of the given quantities are ob- . 
tained even viieri the second tertn is dropped from' 
^ tiie exp re ssio n for the expected shortages, and 
yjhen the last term, is dropped from the expression 
tor expected on-hand inventory. --Ihe given assunp- 
tions and sinplificatioii then Icsad readily to the 
foUwing nonlinear progranmng prcblen (^^iiich ■' 
Sc^irady and Choe (197i) proved convex),,' - ■ 



minimize 
^Q/r 



Z(Q/r) 



N 
Z 
i=l 



subject to ■ 



N 

■ :z' 

■ N 
i=l 



\ 



1 1 r — 



>:o 



CSC) 



with ;^r£ unrestricted in sign, >;.b , 




.and g^^ and g2 representijig the investnnent and . 
workload constraints i respeqtively'. . • . * • 

The prcblem data for the Schrac3y-Choe three- ^ 
item example and the initial starting point for . tie 
SOMT pt.ugi.cm > are shown in Xable .1. . As. iixiicated in 
the table, ♦ the lead-time f^^^^rrvw-^c- ai^ standard devi^ 
tions,. the .item imit costs and' mean demands; and the 
investment and vjoodcload limits are all treated as* 
parameters ia- canducting^the sensitivity analysis. ' 



AC. 



'has the same 



. " , iDable.2 mves ^the joamputer solution and Table 3 - 
the final eStmate of the first partial derivatives 
of thie dptinal value function Z* with respect to 
•tJie jor^ Relative to the*, criterion 

• used in the oocngxiter program, ti^Table 3- results 
indicate' that: the cptimal , : value Mfction is sensi- 
' tive to parameters K2 / c^ / a^^ / C2 / and 

<MaiV infterisnces are pos^le. For -example/ 

the fact 'that the solution, is particularly sensi- 
tive to the values of the standard deviations/of 
.the lead tume demand of itDeras 2 and - 3 might indi- 
cate that/ since. these parameters were obtained by .- 
sanpling, additional sairpling of these lead time . 
demands ney very .' well be warranted to reduce the 
associated standard deviations. . . ' ". > " 

^ iable -3 also suggests that the optimal solutuxxi • 
•value- is yfery "sensitiive to all of *the item costs.'" 
If tiie structure of Problem (SC) is examined, this, 
result may at' first appear contradictory since the 
c^ appear only in the investment constraint ani 

the optimal value function, according to l^le 3, • 
is aj^parentJ-y not very sensitive to the investment 
■limit Ki - ' The problem is one of precise inter-' 
pretation. The partial derivatives measure rate of 
ciiange. But inspection of the investment constraint 
g^ (Q,r) at (Q*/r*) rSreals that the change in an 

item cost c^ by any amount 

■ "effect on the constraint as a change* in the. invest- 
inent limit of -(r^ + Qt/2,- p^) ag^:... Since" . 

the quantiity in'^arentheses may be verified from 
. Tables 1 and to be muchrgreater than 1 for aJLl , 
i , . it follows that the effect of changing c^ 

hy axiy increment ,6^ will be much greater on the . 
constiaint (and hesyx, on the optimal value 2* , 
since the con st r a int is bindii^) thcoj^ith* effect of - 
changing K^" ty the same amount . This irrpli^^' 

that '!3ZV3c^i > |32*/3K^| for'eada i .and,^in- 

::fact, it caii be; shown here that 32*/^c^, 

:;= -(rj + - .p^.) 32*73K^ , so that the relation-' • 

•"'ships indicated are indeed precisely verified. 

rihe alaove observations iriic^t also suggest that, 
some care mdst be"" taken in .intierpi^eting the results'/ 
iiianges- in^ tlae- parameter associated with the larg--- . 
est fin absolute, value) partial derivative .will 
. give the greatest local change in tbk optimal value 
of the ^objective function, oonpared to a change of 

■ the same magnitude in any oth^ peurametDer* taken; 
individually, Thii follows because either the - — 
objective function awVcccscme of - the cooistraints . 
(as above) are most significantly affected by this 

■ parameter change at the current solution > General' 
rules have not been given for selection~of optimal 
dianges in the parameters, i.e:., for determining 
the xjptimal magnitude and o c n bination of such 
<±iai)ges*. It is well, beyq^ the scope of this paper . 
to pursue' this 'tecro-anai^sis" determination, 
thw^ it should be noted that the greatesC local 
rate. of decrease in. the optimal value function is 

^along l±e.directiQn of the negative of -tiie gradient 
"of this function in parameter, space (i.e., alcmg 
-the vector conposed of the negaiu-ve of the ponpo- 

hents of the partial derivatives with respect to 
. -^he various parameters) - A ;user would nonetheless 

have to determine the feasibility of ^this direction . 

of diange". and, . if feasible, tiie optimal ^laove; along 
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SQUTTION AND LAGRANGE MQIJTIPLIERS 
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T^le 3 

OPTIMftL VIWE FUtCnCN DERI\M5VES 



PAKi'iAIS 


ITEM i 


1 


2 


3 ^ 




T -*0000 
.0119 . 
2:i713^ 
- -0012.' 


-.0003 
.0897^ 

1.0345^ 
.0025 


-.0008 
.1729^ 

1.4452^ 
.0022 


32*731^ 
32V3K2 


-•0052 - 
-.6^30^ 



"^Deemed "significant" ty criterixxi, 

[4^2*[/i* > .001 .for a unittitange in the 

given pararreter/ vrfier^ A^2* is the .esti- 

■ • TOted' first order change in 2* ^ Hiis cri- 
tedSxi was seliected arbitraril^far^^illxastra- 
tive purposes* Criteria appropriate to the 
particular application can be selected by a - 
\2ser.; ' _ ■ * • • , ■ ' i 
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this vector, taking into acoount other factors svdti 
as the relative "TOst-ef fectiveness" of any sched- 
ule of -changes inTOv nocSel parameter. , 

•■ Referring; back to Table 2, we note./that i^ V: . 
..Lagrange n«iltiplier u* . .is nuch greater 'than,, u*- 

Iteccilling the "sensitivity" interpretatiori of 
• Lagrange 'niiltipliers, vAiich holds under the pre-i ' ^ 
~'S€nt C9nditp.ons/v it. -follows ^t^ -u* =^ -azV^K^ 

vax)d xx^' r^^^Z^/^K^^ T This conclusion is consis-^ 
:^^^¥'iJtyi^ii :i&^'i^siiltL obtained , in l^fe 3, and it 
iteansVthat -lSie vx3r^ constraiiit- ;'g2' , 'is by far 

tite -more .effective in detemdning the ininirnurn 
nunber of e:^>ected time-weighted shortages at the > 



, e.g. 



■ a.' sirall 



current value of the paraneters 

increase' in K2 will have a greater effect on 
reducing Z* than^a small increase in . 

Nonetheless r a user must* again sirttultaneously oon^ 
'sider. the corparative- costs involved in making 
finite diangesr in conjunction with their ex- 
pected, effects, to arrive at an optimal marginal 
iit^jrovertent based on this first ord& information, ^ 
Tile jsensitivity information is ^valuable, but re- 
guiines some care in- exploiting 

l^le' 4 gives the estimates of the first deri- 
vatives of the cptirtjal reorder quantities Qi ^nd 
reorder points r^ with respect to each of the 
problem parameters. This is extrer^y det a il ed in- 
formation ^-toch gives an indicatibn of hew ,the com- 
ponents of -the solution \'ector itself ^-all change 
as the various -parameters change. In particular., ^ 
this information can be used to obtain a first ord- " 
er estimate of the solution vector of a problem in- 
volving different parameter \alues^ having obtained 
a solution for a given set of parameters - 

The partial der±vT'^ tivies of the Lagrange multi- 
pliers with respect tai the parameters are given- "in 
.7^1e-5.— .^5ga±n,^e5e-c3cn"be'u6M to obtain first 

' order estimates cSf the J^agrange. multipliers . of a 
problem with, different parameter valjies. In^ par- - 

• tictOar^ ^the relative effects of the constraints : 
on the optimal value of problems involving differ- „ 
ent parameter valuesJcan be estimated- Furtherr 

aaore, it can be shown that the partial derivatives - 
of the roultipliers with respect to K^. ' -and ' IlJ 

yield the second partial derivatives of tlie op- 
tiiiBl value"SjHctipn witli . respect to the param- 
eters ■ and K2 , under the present oondi- 
ticns. Thus, the kind of information given in 

".lable 5 can be: used 1^ provide a second or^er 
estinate of the c^Jtimal value^ function Z* tor 

. different valLJes of these 



Ito illustrate and test the explication of the 
type of information provi*^ here, the first 
tial derivatives with, respect to .c^ of the 

nal value ,functian Z* , the solution cctiponents 



ai^ 



and u<. 



and the Lagrange multipliers^ 
xrfere u^ed to give a first order 



^i 



(Trior's Series) estimate of' the corresponding 
solution values associated with the prd^lem ^*^:iere" . 
the given value of c^- was increased by one. dol- ' . 

lar. Ihese estimates were cditpared^ with the re- 
. specJtive values of the solution obtained by actii- ^ 
ally solving the p er t u r b ed problem'- The results 
are sunmarized in T^le 6. Though the -perturbar. , 



tion is large (the par^teter being increased by 
100% of its airrent vs^fiueV/ the estimate are seen. 
to he ex t remely accura& with the exception of the • v 
estimated reorder qua^^ Q^. Many uses.;Could. _ 

be made of "the estimated solution; e*g. ,.:,shoald".it ;^2^ 
be desirable to solve the «pertrpbed pfc±)le^ ^ 
-ately, it would be cctrputationaLLly extremely advanr / 
tageous to use the estimated - solution as a starting 
point. \ J^ '^ i ' m » ' 

The- ccnplete-e^qjioitation of tMs., sensitivity 
analysis inf ormatflon now available will . dsspend, 
largely , on user interest and' ingenuity.*' .-.^ 



Table 4 ; 
rCN POINT SENSITIVITY 



PARTIALS 




ITEM i 


1 






ar./ajEo 
,1 ^ 




- 18.7610 


- 14.9065 


^^.2265 


. 6.1961 ' 


9 ^ c 


3Q^/ac^ 


/2O8.8688 


15^3140 


14-3755 


ar^/3c^ 


- 31.7918 ' 


- 10.3425 


- 20.0020 




- .8469 


* .2271 


' ^.1084 


3r-/&02 


.1273 


'1.0337. 


.4919 




8-1908 


- 4.9719 


3-8522 


3r^/3a2 


2.6783: 


'-^ -7.2676 


- 7.1087 




- 1.2374 . 


- .4033 " 


-5^43 


3r^/3a^ 


: .2611 


- ..3839 


^ . -0446 


aQ./ac3 


- ■ ' :i670 


.4442-^ 


- -4251 


3r^3C3 


, - .2.3072 

r 


. - ■ 3-1702 


.- .12-1523- 



. Table 5 
L.M* SENSITIVITY 



QPTIMAL L.M- DERIVATIVES 




OCNSTRAINT i 




1: INVESTMENT . 
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. -.00.02 




-.1382 




'.0006 




.1635 ' 




, <0000 
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2r 2 


.0002 >■ 




.0489 . 


3u^*/3o^3 . 


.0000 . 




.0020- 

*■ 


9U^*/3C3 - 


-0003 , ' 

■ * ■ . 




.0323 . 
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INCREASE OF PARAMETER • • - 
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• ■ -The Generalized Inverse in' Nonlinear Programming" 
- : . Ecjuivalence of t±iiEt Kuhrij^ and ' ■ ^ 

Generalized implex Algorithm Necessairy' Conditions* ^ 



L. Duane* Pyle^^ 
iversity of Houston 



1;' 



Introduction 



\. The^^qui valence of the Kuhn-Tucker . 
and the .Rosen .Gradient Projection condir 
tions for optipiality .yas estaODlished l^y-^ 
Wangasarian\:, { 4] ; fwho considered "the 
following nonlinear progri^irmiing p^^ 
formulation: 

M£Dciinize f (x) 



(1.1) 
where . H(x) = 



• m 



> 6v xcB 



and l\^(x), > h^(3c) and f (x) are 

concave, dif f erentiable . ^ 

Ce IS a vector of "zeros.) 

Except for minor changes .in : 
notation, the following, alternative, 
nonlinear programming problem. . f ormu- 
-latioh* is discussed by Luenberger [ 3] : 



where 



Minimize f (x) 



H(X) = 



(1.2^ 



= e 



G(x) 



> e , xeE 



,n : 



and , ; > :li^'(x) , ^^^i :r^s+t' 

and f (x) are dif ferentiable. * 



where 
(1.4) 




: ^(x) = 0 for 
, g j. (x> > 0 ' for j ^^2- 



and J;^^2 ^s+l. 



s+t} 



then the constraints (x) , . - , 
and qj (x) for jeJ^, are -said to be active^ 
at X ; the constraints g . (x) for jc J- are 

said to be inactive at x. , A point, x, 
. satisfying /CI. 3) is said to be a* regular 
: point for the -cons train ts (1.3) if the ^ 

gradients of the constraints active at x _ 

are linearly independent. . 

The ^Kuhn-Tucker Conditions [21- are 
satisfied at the point x for the nonlinear 
programming problem (1.2) if "there exi st 

vectors w^^^ and w^^^ such? that 



Cl-S)- 



7f{x) + i7h^CxJ 



. Vh^(x)l w^^'V - 



where" fw^'^^ , G(x)) = 0, w^^^ <. 8 and 



7v(x). = 



|2-(x) 

3Xi 



3v 
3x 



(X) 



:;>;/s .. .. 

V " Luenbergei; [ 3] prove's , the ; fol lowing 
.(necessary condition): . • ' / 
' ' - • . ' ' ♦ • ■ ^ 

Theorem 1.1 = Let x be a relative minimum 
point for the probleii> (1.2) 
. and suppose 5 is a regular 

point, for. the constraints^ *. 
, /" : * r. "• • ti. 3) , then^ the . Kuhn-Tucker 

: ^ conditions (1.5) are psatis- 

f ied at x. ' 



' '^^^^'if X iis a point satisfying the 
constrafnts. * ' 

(1.3) H(x).= e, G(x)''> e , 



. . • OlFm .=; [vh, (X) ; ... vh^ (X) ,• 7q . 

^ ' vq. (x) 1 be the-ii-'by -(s+p) ■■ 

■ ■ /' ' ''^V'^ ' ■ *«^V^7- 'V 

. matrix ;whose Columns consist of the ' \ 

:gradiehts tothe cohstrainjts which are \ 
, active -^t x; that .is,. = {j^, j2V-.-^;jVW^ 

. -The, Rosen Gradient Projec tion:Cohditions \ 
. [10] /til] are satisfied at the ooiht x sfpr 

• -the nonlinear" programming , problem. (T.^2i -i^ . 

-,rr.-rQ!:/x).tl(^)] ' vf (X) = :6 ' ^■ 

;fi-^>*and^^^^ ''y:/-^-:.: • ' ' 

l((C(x))'^ Vffx)]. V 0 for all J^/ - 
^ ^ / ^ . ' ... 

♦ . .where CL i s the_^ ge neralized inverse, of , 

;<'.:The^.'ef^ress ions analogous to (1-7) 
formulated by Rosen, Manga sarian, and 
\Luenberger may 'be obtained by making use 
of the relation ^ . ^ . . . " 

ci-s) cL*'f^)'T= [^i(x) aJ^(x)]"^a(x) 

as; for example, in . ^, . 

= I -of cx) [a(x)^^(x)]"^af^)- * 

, - ' . ■ ■ V - 

Relation (1. 8) provided the basis -for ; - 
';the computational, approach originally pro- - . 
.posed by Rosen [10]^, who ^ gave a procedure 
..for -tupda-ting" K^(x) OF-Cx)-] "1. Alirexnatives 
proposed: by ^ill- and^Murray [ 1 ] and Pvie 
^7 I / ■ ] T^" ] ^ employ an orthonormal 
basis -for the null. space of Cl^in obtaining 
'a representation for- 1 - Cl^£?.. wdtivatioh 
for the developments, given in [,9 ] was 
provided by the 'results of numerica'l 
experiments, inyolvin^ rand-omly generated - 
linear -proqrammi-nq. problems,- intexrpreted 
in the 'context of the geometry of. 'the 
-simplex- algorithm. ! This .interpretation " 
has led -to. .the extension '^nd .refinement ' ' 
* of the reisults given'in [ 7']. and [ 9 1 - as - 



. reported in ^ [ 9 ] 



-In .this- paper two results are pre- 
sented. 'The. first, provided fot complete- 
ness, is -an application of ilanga sari aria's, r 
approach; demonstrating the equivalence^ / 
.of the 'Kuhn-Tuckef . and ■ the'. Ro-sen^ Gradieiit ' 
Projection Conditions as formulated for 
the' problem {1. 2> - The second resiilt 
• establishes the equivalence of the." 
Rosen- Gradierjt Projection Conditions 
(1.7) ."and certain conditions which result ^ 
from a ^ Uiatural extension of a general i za- 
tion^of. the. simplex algorithm [9] to the ^ - 
nonlinear programming problem; 



. 2 * * ' Ec^uivalence of the Kuhn-TucJce$>' Condi— 
tions ^and the. Rosen Gradient ProjGC- • 
tion Conditions . vw. * - 

..r-y-^^. — ^ — — ' ;.>>^-v-, 

■<,■■ " •. ; Suppose. j^l" ."7) hold.s at some point x ; 
■ which, is reg^ax for the * constraints. .(]^.3)\ 
*Therf . \ - 

' (2-01) " [I"'- a'^{h:'au)l vf (x) 

[.I - 0^ ikUct {^^^^ '^^ 

.. ■'■■implies- ; • ■ ' .' 

■ (2.02) vf (i) +.<l^^c5) 't-{a/(x ^ e 

Define w^-^^r w ^^^.as.';fpl;low^s■: 
(2 . 03) ("w'^^-^H . V -((V (x))?\vfJ^) r 



^2) 

(10-:. 



where w""^^ . haa^ s ',elements 
p elements- 'Then ftom *- 



^^'^^L [(C^ (x) )^' vf (xyi^^ for ali jcJi- 

•■* — f 2 V J ', 

it follows that w^ < e . Now define 



(2. '05) w^^^ = 



- C2) 
w 

-e ^* 



where w^7^ has t elements. (Note: G(x) 
in C1.2)(^is composed of t functions) 

^s w^.^?l c.^, and from (2.''02.) 
and' ('2.03)and''t^ie form*of w(2) it follows 
that ■ 



1' f -wf^n; 

(2.06) .^^) ^ d^{K) [; ^j(2)*| 

(hi + j.o?ix) ,a?(x)j^ j^'^J.^^^ = 



Vf I 



= 8,. 



where CI (x) is a. matrix ^f ^ the gradients. 
7g. (x) for j'ErJ2f: corresponding to inactiv^ 
constraints at (2.06);. together wi'th 

C2) •,r.;(2)* ' ' 



, (2.07) . GCx)) .-/(^^^^^ , 

where w^^^ < b', is\ (1.5) 



G(x)) =.0 



■tow, suppose (1.5) holds at some/ 
po.int X which is regular for the con- 
straints. (1.3) . For simplicity, we.. " 
adopt, the notation^ ■ ' ' . 




;(2..08)- ' jj =.s .+-l, 

.' wheire ^1 ■ = { j 1 , •'•'f^ jp> • 

Letting 'w anH w denote the vectors ? 
'which satisfy (i-Sj ,. the relations 



(2.09) 



,(2) 



<_' B,: G (x).' ' >_ "9 ' and 



(w^^^-, G(x) ) =0, ■ 
taken - together, imply' that - 
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A at 



thus 



Where 0 and 



has p elements* 



V {2.11). 7f (x). rvhj (3c). v...-7h^*^(x.) J -w^ 



-or. 



(2.12) Vf( 



(x) + a^(x) |-(2)j 



e - 



Since x is a regular ^poii^t for the 
•coHstraintS' (1-3) r (x) has full coliimn 
r^nk, thus 

■ « 

therefore. 



= 9 



; * - pr, sinfce w^^^ l^r ' 

r(Cif*(i)')'^^Vf (i) ]j > O' for all^cJi 
Solving^ (2.14)' for and 



^stibstituting in (•2-12) ^ obtain 



•{2.16) 



7f(xJ - a^(x) (CiJ'Cx))*^ 7f(i) 



II -a'*'<xOa.(x)] 7f(x) = 9- 
' ^ (2.15) and J2JI6) are the Rosen Gradient 
Projection Conditions (1-7) , formulated , 
for' the ."nonlinear prograimning* problem "(I- 2X. 



" and h 



rvv.,.(x), 



v.:-^ (x) 
s+r 



XGE 



3. Equivalence , of the "Rosen Gradient; 

Projection Conditions and^^ the Gener- 
■ alized Simplex Algorithm. Conditions 

^ Consider the following nonlinear 
•programming probleflf formulation: 

- ' \ ^ ■- . Minimize fCx) 

*hi (X) 



where 
H(3C) 



= e . , 



(3.01) 
•*G(x) 



• Fv(x)1 



h j (X) ,N 

v.^ (x) and\f{x) are- f ferentiabie. 

f^ote thcat .art^ prablem of the • f.onft ^ (l-'2)' 
'%ay be re'f orraul^ted\in "tHe form (3 .01) 2) 

. it is understood' ttj^at V (x) , "if not . 
"empty,: consists of all the "coef f iciented** 
^inequality .constjCaints; .that is, vr^ (x) y x^ 

for ( j = 1 , 2, ... n) - ' . 

Suppbse X is a regular point:, for 
(3.01) and, for notational simplicity, 
that the active constraints, Vj (x) from 



the set V(.x) = 



|^V,(x)J ^ 



6 consist ofdthe 



constraints v^(x5 



for {i = s /. m) , ^ 

and that ,the active constraints from tf^sk 
set .1 x ^ X >^ 9 are the- constraints ^ 

(u^^', x) = O.for- t j = m+k+1^ ... V n) , 
where (k =, 0, lr_.^.- / (n m) ) / and I is 
the n-by n identity matrix composed of 



un^it vectors 
(Note: The /special case k = n - m, j = 
n+1, n corresponds to the situation where, 
x^. > 0 for j ^^lr2, n.) . 

Define ' ' ■ \ * - 

(3.02) A^(K) ;= [Vh,(i).;:vh^(x)vy^^^(x).;. 
Vv^(x) ]/an n'by m matrix whose columns con- 
sist of the gradients to the "coef f icient- 
ed" constraints which are acti^^e a-t x^y# . 

(3 I 03) c-(x) = 7f (x), 

(3.OO'' b(x) = [A(x)?x.^ 

and .tberuij^note the columns of A(x) as 
follows:' / '■ ■ * - • , 

(3.05) * A-(x)' = [p/Dc) .... P„ fx)]. , . 

Consider the following l*inear program- 
.ming problem approximation of" (^'•Ol) at x: 
Minimize (x^* c (xl ) ' ^ " " - • 

"(3.06) where . ^ ^ •* • 

- [A(x)]x = b(x) ' ■■ 

if > e. , 

The developments given in [9 J ' for the 
linear programming problem may be applied .. 
to (3.06); ' :v ' 

In suiranary,' letting - ' ■ 
(3.07)- '^(x). - [Pi (x) ... P^"(x) ] , X a regu- 
lar-point for (3.01} ' implies B(x) is non- . 
singular,. Forming 



- .(3.08) IBC5c) r^(^)^ I^'^m+l'^^ - - ^f^JlJ. 



obtain an orthonormal basis, {yf^^'^^l^x) } 
f 02:> <i » 1, 2, ... , k) , for the subspace 
in which is spanned by the .linearly ■ 
t independent columns of i;he foll'bwing. 
roa-qrix: / ••■ > . *. ; 



where, j « in + k, n 

- ^ -2^ . . ^ (n — m) 

"and the *'j"th element in the vector 



r-. 



rR(x)1 



■i . 



1 

0 



0 

1' 
d 



6 



fere R(3t) '= -/tPjn+i (3c)V,. .P^^j^(xn -is m by 

k^ ■ ^ i^^.by k^and; 0 is [n-(m+k)^] by k. 

^ - . The Btfeen- .Gradient Projection Condi- 
tions {l.J^ formulated for problem" (3.01) ' 
in accorqance with, the notational "^conven- 
tions- assumed above/ involve the matrices 



C3.10r CLM = and I-oTcxOi^Cx) ^ 



(m+k+l)_ ,^^ uCn)^] _ 



where XJ = [u 

-■ .■ ■ 'Now, Cl(x} F-{x) = O^by construction 
and the k columftis, of (x) f orm a basis for 
J:he null space. of 'CC^JC) / thus ' " . 

^ ' ... ^' . jc Cm+i) Cm+1)' . 

^3.11X I^a (x)£l(x>.= 2^ n (i) n (X) - . 

i+1 ■ 



^ is unity. . ^ - ' " ■ ^ ■■ . \ 

Then , 

■ ■ s , ' . ■ 

, (3.14); a. (£>+(x))y-.c(x) ] . 

■ :.-for.. j- = 1, .2,:.,*. -m-'. ' ■■. 

\^ ^ It 2, . . , (n - m + 11 . 
and 

k ■ ■ ^ ■ . • 

^ k- = 1, 2, (n ^m) * . . 

Proof: Upon' substituting. ^t^(x> ( ot<x) 

. for Q,'^(x)Cl(x) in (3.12) and (3.13), 
•respectively, 
obtain 

k 



The developments given in' [ 9]- are thus seen where j ^ 1 r 2, 



..r - - , + T ^ •^(x)u^3^ 

. (3.I6J Cj= (^Cx)(Cir<x) )^ctx) J 

= ((-£t(x))'^c(x),£u^^^j:). = 

[(£it (x))^ .c (x)] 



(j) 



„ to provide- an alternative method for ob- 
taining the orthogonal projection matrix 
' * ■ ; - [I ^ft^'Cx).a(x)] . 

. The equivalence of the JRosen Gradient 
Projection Condition, r ^ - ^ 

[(A (x)> Y^vCx)]^ > 0 

'emd the analogous . cpnstruction obtained - 
using, the generalization of /the.. simplex' 
algorithm given in [9], follows from ./ 
theorem 3.1. V ' / 



.m 



k 1, 2, (n - m - 1) , 

which is (3.14); and - 



• -^S ,17 ) c j J ^ ^ J J (X) V 



-p . 

e 



2^ 



3 J 



. .Theorem 3 .1 : Let 
(3^.12) c,. = iOt lx)CHx)c(x 



) ■ 



wuere ' j = 1, 2, ... /"S 

. k - 1;. 2, . . . . • , (n - m + 1)' 

■ and ■ ■ ■ . 



-p . 

■ \} 



) 



, where .j. = k, . ... ,1 n 

k.= 1, 2, / (n - m) ^ 

which is (3'. 15). « 



■1 

6, 



j _ 



) 
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I 



jSinarks: 



<i) The>c^s{«"'<f"^'-:]^ 



and 



r - "1 

-P.-(x) 


\ • ■ • ■ 






. 1 ' 









-B ■ (X). p. (X) 

A- 



. . bo.tih involve *B, ; (x).r * and this peirmit^ com- ^ . 

' outatibn of the %' Values by a ■ Revised*. *' s 
:•• ; / . ■.. - ..••..^ '"^ 

Simplex Alsgbrithin. approach/ as. described 
. ' ^'xn; [ -9^1 J -AlsdV se/e [ 5] wh^e the matrix 
; FCx); is used , direcvtly in implementing the* - 
'^ method 'of reduced gradients - 

(ii>* In ( .9 ] the vectors "f ti ^^^^ }* for 
•.(1=1 -J ^ * . ^ 1^) were ^obtained sequentially 
by « application of the Grfam-Schmidt . ; ; 
orthogonal ization procedure. From (3-12) - 
and (3.13) r- it follows that the individual 

Vectors,, n ^ t are 'hot required\;in realiz^-- 

inq the Rdsen^ Conditions , thus any other ' 
'niethbd (e.g. :[ 11 ) fcrr" generatii^g tJie 
orthbqonal projectipn having -a range -equal ; 
.to the colujnh .space of r (x) . could be 
utilized. '^J-^ ' ' . ^ t s 

Ciiij Note' that in. (3. 12)- and (3 .16) , : u/^^ . 
denotes an' m by 1 unit vector, whereas in 
(-3.17) u;-^ denotes. a.n n by 1 - .unit' vector. 



." [8] pyT<=> ^' L^D> .. A Simplex- Algorithm- ' v 

• GradientlProi^ection'^ Method for Non- 

' linear Programming , CSp TR 55, ^ 
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TEAtfiiNG MATHE^lAflfcir. PROQRAMMIXG 
\ ' TO tflE CONSUMeIi 



■ .. Marshall- L. .Fisher 
Decision . ^ienc^s Department 
The Wharton School. 
University of Pennsylvania 



^ Most courses on ntathmatical 
programming, are 'taught'' by producers 
(i.ew i;esearchera who are developing 
mat^matic^l ^ prpgranuning' technology) to \ 
potential ^consumers of that technology. ' 
The pedagogical approach that is mo'st 
natural for a producer may not be at all 
natural for a potential. cansSmer / a* f apt * 
that explains r in part, i,why, many 
• potential consumers fail Ito become^ - 
actual consumers. This jpaper " is 
concerned with the par ticul Jr ^heeds of 
the/consumer. ' J. ' 

> T will, provide -a number of .Specific ' 
suggestions*, for course content ;and 
design. These Mggestions are based 
primarily . on* my exper«iences teaching , a - 
fundamental course. on mathematical 
methods^ for* decision .making off e*©d<:fey 
the Decision Sciences Depar tment':ci5 the : 
. Wharton School. This course, ' which 
devotes 10 weeks to ^linear and integer 
programmingf, is. taken , by about- 200 of 
the 600, students ^hat enter the M.B. A. 
program , each yjfiar . The backgrounds of 
the students vary widely. This year my - 
twp sections had a total of 70 students 
tha-t ihcluded 12 math - majors, ^^0 
engineefrxng and science majots, -iS 
management ^nd economics majors and 10 * 
majors in yarious non-t.echnical areas" * 
like English, Anthoropology, Philosophy 
and LMguage. . Most of these students^do ' 
not cucxently plan . to become 

•specia-lists in Mathematical Prograimninq, ' 
Many have nbt yet decided on a majpiS . 
but of . those ^o • Kave 31 plan"\to" 
specialize in Finance. . i ; - 



Those 



,who preach 

tenets. CertaxiTi 
constrained optimization . pr 
useful ^ canc^tual ^franjework 



mathematical 



programm.ing^ should " also practice irts' 
CertaxMy the viewpo/nt of 



^ij^es ■ a 
>r / the 

problem '^ad4ressed^ ^ ■ j this 
paper— 'designing a'^ couzfse .•*on 
mathematical programming for a/^pecif ied 
purpose .\ I will use this / framework 
informally la developing my snqgestions. 
I will- take as'' an objective^, that ^.the 



course should provide maximum benefit .to 
tne future careers of the stud^ts. 
Benefits caa range from jchanging the way 
the students ttjigk about their work to 
motivating . alW : enabling* them to 
f^x^mulate and solve mathematical 
prag-rammmg mode'l-s x>f problems forced in 
th^ir careers A "Constraints are impo-sed 
on the >actiyities w:fe can. perform to 
optimize this objective by^. the limited 
teaching time available, by the students 
diverse, and in 'some cases, limited 
mathematical backgrounds, arid^^by their 
long term career' plans^that freouently 
■ ^ Pi^ce imary" importance ^' on 



technical* expertise 
programming. ' r 



niathematical 

. ■ 

list seven 
design of ^ 



of ^ 
optimizatiiSa 



r - will . now 
recommen tationsT for " the 
course that "solved" the 
problem." I ^have just outlined. All 
suggestions are admittedly subjective, 
but X believe they follow logically from 
the framework I have civen. 



be 
"fr 



Me.thodological : tables shoald^ 
' sel ected to maximTze tne ratio o^. the 
short r'nn Feitefits to TFeir cost: . 

By 'short run benefits'' jmean thfe 
ability -to operationally solve ^ 'real 
probleiiLS and - ^by , * costV f , mean the 
intellectual prerequisites' and\. time • 
requirements to understand the topic. 
Thi's^ criteria leads'to the selection of 
topics li.ke , Minear programming, 
seperable programming^* heuristics and 
branch and bound . methpds for integer 
'programming models', .^and tjie special 
algorithm fQr ^ahspbrtatipn problems. 
It requires - the exclusion of many 
.fascinating .^topics in nonlinear - and 
.integer programming that (a) would 
reguir;e too much time%to teach^ ,or (b) 
have' ^-'too • great a -mathematical 
prerequisite, or (c) lie. o^n^^e frontier 
o£ research , and thu^^fave not "yet -, 
demonstrated their practical usefulness 
or (d) 'have all of the above drawbacks. 
1 would -"list" as borderline oases 
Lagrangian. relaxation and an infoi;mal 



treatment of the analysis* of heuristics 
and algorithms for ,combinator ial 
-opt-imization problems.* * 

2. • Algorithms shoold be taaght in a 
way that conVey~the ir^og ic but avoids 
techn ical ities; ' • 

, ' •* 

One good method foV doing this is 
to simply demonstrate the algorithm on* a 
small example derived from a real 

•approbation: • The students "usually hav^ev. 
a'h intuit i-on about the example ' that "j'caiiv^ 
be used to explain, the . looicof the 1 

Algorithm* - This excercise ;can be 
tollowed , with a verbal statement of the 
algorithm for , the general case. 

A variation on' this approach is to 
present the students with an examnle 
ask them to use. their common seasc^ to. 
develop a solution procedure. 

-Pjre'guently. the students will be able to 
discover the esse,ntial features of a'n 
algorithm themselves and at a minimum 
this experience- - provides a good- 
perspective for- understanding a ' nore 
formal presentation- of the algorithm. 
For exa^pler whet?^ confronted with a 
problenr of shipping amounts of a product 
from several supply points -'to several 
demand" points most, students will ouickly 
devise a tableau. .format for ; recording 
possible solutions'" 'and discover a/ 
feasible solution/Gsijig some varient , 
the - Northwest corner rule.;\ v With a ^ 
little coaching -they will Tnake improvdng ' 
adjustments in, this solution until "an 
optimal isolutiort is obtained.^ It' is 
then only - necessary to show them that 
"their" adjustment . procedure can be 
systemized ,and to p^*vide^an optimality . 
test and proof. • - * v: 



3, 



Use 



articles 



that 



r epo'r t 



appl ication - saccess stories to show how 
■ ma,thematical progr amm ing can ^be used i 

' '■ ( 

The - increase^d emphasis on the 
"publication of . solid application work 
^ has produced^ a. wealth of articles that 
are" highly . suitable _ ^fbr classroom 
.discussion. . " tTournals -'like Internees 
are 'ideal sourbes for. articles ofJr-this 
type. Recenfe examples of ^applications 
articles *^ iiv . this^ journarl include a 
d iscussian' of inventory planning' far a 
-ininia^r * company ; (ReddyI6]) and 
description of how to* win\ an. - election 
with linear pr^gramiping ^{Sb^ithwick and 
2i'<:>ntS[7r)-an article that iT? sure to be 
a., hj.t in ■ any ^ year ^ divisible-, by 4.. 
Management Science / Operations Research 
and . the Harvard Business Review are also 
/.not ' .to be \. overlooked . for articles 
"suitable' ^pr ^this purpose. The paper 
Klingm"arir\-. .Randolph "and Puller [4^ is 
highly : recojnmended • , for av real-life 
"^illustration-of branch and bound methods 
combined ^ith transportation or network 



comoi^i* 



Vtlow models ^ for a location ■ analj^s'is 
problem. ' G€offxion*[21 . gives . an 
^^^xcellant managemen t-or ien ted d iscussion 
6f integer programming method^ ' .^^^ 
warehouse location. 

^ . ' " ■ ■ . ; ' --^ . 

^se cases to* develop model ing . 
skills; * r ^ 

The ability to construct a 'model • of. 
a complicated process is a' skill best 
acauij;ed through practice. Case study • 
probl'ems are an excellent way for"^ 
providing students with this practice if*- 
the cases are appropriately selected. 
The case should present a comolicated^ ' 
messy, and appar en tly . baffl ing decision 
problem. At the same time^ the problem' 
should have^an underlying structure and . 
sufficient 'data should be included to' 
allow the student ' to d iscover and mo<3el 
the structure^ and eventually arr ive at a- 
'satisfying resolution of the^original 
problem^ The feeling of haying "created 
or.der out of: appar ent , chaos that ' a ^ 
.student experiences from analyzing such 
■a case is the single thing'moGt likely, 
, to 'turn him. on' to the . ^s^ of 
mathematical programming. . -\ 

Errors ' of. ' two types, should //be 
avoided in the 'select ion of cases. ' Many 
cases are too structured r > really just 
expanded formulation exercises. Others 
" that 'have been designed for policy 
discussion are too unst^ructured and 
don't ' provide enough data ^or the 
student^ to sink his teeth.^nto. 5ood v 
sources for tases that strike a balance 
between* these extremes are books J.ike_ 
. Berry and Whybark[l] s-and Von 
Lanzenauer [8] , 1 ' have also had - 

Sconsiderable success with ceases 
developed from my own or- another faculty 
:■. member's consulting experiences. 

.Students in. the. Wharton ' course 
.described earlie^ aii^yze cas^s in" 
teams. Each^team is s^i^ted to .have 
some students .with teplinical' bacJcgrpunds ^ 

^ anS some 'with non-te^nical backgrounds." 

' Th'e results of each team analyses , are 
• pre^nted in a'* written ireport and an 
oralu— *«porti to the class. The Wharton 
CoitOnuni-cations Consultants^ a^. service 
group at Whartoiir orovldej expert 
coaching and video taping equipment to 
assist^ ^the,^ teams in their /^oral- 

■ presentations.^ ' - • . ' 

_ . * - " .-^ _ 

' 5. ■ . Provide computer codes so that case 
analyses may be pars'aed to comple?tTon; 

\ The computer codes shouldz>^e easy 

to 'Use yet^ st^ll solve reasonably lar^e 

oroblemi^. * Ideally ; they . .should' be 
available on' an interactive computer.^ 
We hav6 had .excejL lent success with the . 
patfkage" of, progranfs described in Land 
arid Ppwelft^]. The # package^ has an ; 
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.Or- o 



li^lir '"?nL« provides. 
_._fA".^^aA»„-._-.in.teg.eiL, and --parameter 

progcanuaing capabilities and is^ovfled 
n%r inf 8*"<3ents interactively on a 
DEC-10- computer . 

6. Bring in Ogtside -^SDeake-S' 

foTtEe^lieaf!^^ ~ 

groups and * consultina 
organizations are freouently involved 
interesting applications of mathemaScS 

JaJk^'rS^C? '"^^ are cuite'h^lpj'to 

talk about them. Preouently these 
organizations are. also involved i? 

sSakL%'"^o"^''" ""-""-P"^ andt^vieS 
speaKing to, cflasses-as a desireabl*. 

supplement to these activitislf^^^ey 

c^n provide ^breadth, bacKground 

i;Sd""'th!^ ^ for the^ ?eal 

world that IS. . unavailable from other 
sources. Examples like that bf nnf 
company which routinely solvll a lo oot 
variable 10,«000 constraint LP^nitsel o? 
iiL^^-'^^i^*^^ activities have a d?Imati| 
impact and provide additional evidence 
that mathematical programming iJ JJt 
only useful but used.' Ideallv - 

^tlltt^'ro .^^^^'.^ chosen''%1L^id 'll 
cSjs used. ' -PPl^"tions article, and 

Z- ' g°P^^^^'e ' the , intuitions ' about 
I^Ei^ systems, 't hat. mafl T e matiHr 
programm mc^ can revealT — : ^ 



usually 
and an 



^^4:^-^°^^^^'' systems ' are 
difficult to understand ' and an 
■tr'^^fnt^-J^ approach- frequJntl? leals 
to intuitions that are 'incorrect 
afior??^'"^ programming models aJi 

fioSt l fon^r- ones intuitfbn 

cboot a conoiex system if. the 'model 

solution is dissected 'a posteriori' to 
. determine whv' a given ^rx^tZ^ 
Obtained. .. (see Ge?«r ion [I] ^"^lor 
discussion and examples on this point). 

There are at least two reasons whv 
a jnanager should develop good in?Citi?ns 
about a system he manages! Firstlv ?f 
nf.f • ^ *^he intuition to underJtlJd'^Chy a 

So^^Sti5L *^'sec5n^d!y/?"thf^^^-i?ioV'l 
JiS^i'^'fr'^^'" "-^^^ so1ution'S°SI 
SeiLilns ^^h%^!-?ela?l to^^%he"s-yit^e1 
^.SSsT a?Me ^- Ls 

t^^^'^'- tSl'Tu^e?o^s"':nl • 

w''°^ ^"«=i"<3e explicitly in the 
model,- Moreover, even for decisions 
Which 'are included in the model 
results -must often be manually ^fdjustel 

intuitive understanding of tSei? sylJe" 



^ J^" -^-^IJ .f^^^^^ 

cnese acaditional decisions. • 

Students will aopreciate this 
?k1"^S"°" if they are iead to liscove? 
th^.^ei^spn^^ for answers' obtaineS^^'S 
^ a^f ■ ° applications articles ■ 

'.S'thf"f"^'=^"^*^"'P^^^ ^" numerous. tli • 
OK ^ transportation problem; the ootimal 
Shadow prices can reveal the initially", 
• surprising fact that shipping mJJj 
^^^^ " cases ^edu« 

' mS^frS^^^^r^- "^^^ programming- 

- model described in ReddyfffJ lead to tht ■ 
discovery tha't customer orders should be 
satisfied . with relatively low orad^ 

FeSenS: 'Sr ^^^'^ gener°:ted^"s| - 

? f , • This ' policv 

, completely contradicted existiSJ 

- v ° «=°"^<=t because of 
scarcity of high gtade product. ^ 



the 



s^.,^on^ experience has ' been that 
^^"^^"ts are much more satisfied with 
^etholr can' r-^^^-^-^tical Vprogrlnming 
.methods can enhance theirr own intuition 

v?fw^o?n?"?? T"" ""^^y ^" «ith-a 

lik^^ ^ho • P^'^trays an algorithm 

like the simplex method as a giant 

tfoS ^^'^^'^^ ""^^y cangoafouSd 

tightening loosei corporate nuts. 



/ 
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ON TEACHING LINEAR PROGRAMMIN(? FUNDAMENTALS 



jJohn _ M . Mul vey 
Roy D. Shapiro ' >/ 
Harvard University ' 
Graduate School of Business .Administrat-ion 

\ 



Abstract 



This paper provi'des an overview of 
the methods employed by the Harvard Busi- 
ness School in teaching linear program- 
ing. Specifically, we deal with the ■ 
pedagogical aspepts of teaching" linear 
programming fundamentals- to predominatly 

■ nonmathematical MBA students. These stu- 
dents will eventually become decision 
makers in government and business, and 
since many of their decisions will re- 
quire the use of cpm^lex mathematical 
models, it is crucial that these indivi- ' 
duals thoroughly understand t?he f unda- 

. meptals of* the application of quantita- 
tive tools. ' The cornerstones of our 

. pedagogy are presented: - 1) ^the use of a 
flexible interactive linear* prograiraning -\ 
system, ..2) the active participation of 
faculty' and students in clas^ discussion, 
3) ..the simulation of semi-realistic " 
sitiiations^cases) in whiph licear pro- 
grcimming has been employed, and 4) a 
'cetrefully prepared set of eo^bi^se mater- 
ials. "/With these element^: jwe believe 
that learning is enhanc^^^^^' that 'the 
students are well prepa^r^^tds success- 
fully use ,linear prograiming 'jiechniqu^ 

* A detailed description* of the ' program 
is. provided. . ,■ ' ■ ^ 

Although quant^-tative ■ techniques 
have become an intfj^xal component of 
most: MBA curricula/ j there have been few 
published papers Wdich deal with the 
■pedagogical 'aspect^ of quantitative 
courses. , To be; svwe,'\there have ^>een 
numerous articles general management' 
science educationV-^or example, .the, 
special issue of Management Scieniae on .* 
^Educational Issues :[2J , Wagner.'s talk \ 
on the ABC's of OR {[6], and several 
papeacs [1, 3, 5 J which ha^^e described , . 
the content of variious MS/OR and related 
curricula. By and jlarge, however, these'- 
. do not address the specific issues of . 
pedagogy- . 

' '* • . . *- 
Pedagogy is particularly relevant v 
tt>day fer several reasons. Pi-rst of all',,, 
the f;ft*Ld of Tnan^agement* science, includ- 
ing, linear programming, is entering a 
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.-maturation phase in which fundamental con^, 
cepts are being refined and polished. Ap* 
plications are assuming an increasingly 
important role. In addition, many of t:cm-' 
orrow^s decision makers who "will assume 
• positions in-' which they are -able to make 
use of analyticar tools do* not possess 
mathematical background or inclination- 
Thus, we might lose important opportuni- 
ties for applying management science^if we 
do not, or cannot, teach non-qi^antitafcive 
decision makers how-to properly utilise 
these analytical tools. See Healy [4]- 
If jnanagement science is to continue grow 
ing, we must develop i;eliable methods for 
communicating it$'; ideas and tenets. 

The lack of a suitable pedagogical 
fr^amework is most noticable when discuss- 
ing the teaching of etnalytical tools to * 
students who ^ do not possess mathematical 
background. This article attempts to fill 
this void by addressing the specifics of 
teaching linear /programming f undameiltaJ-S 
to non-4uantitatiye MBA students.* Herein, 
we use Che pedagogical device of inter^ 
active exposition; a -technique which is 
repeatedly employed by the authors in the 
classroom: Although this report does not 
strictly follow the "case" format, it d^es 
describe the^way. in which lineSr program- 
ming is taught ^t tjie- authors • institution, 
i.e.. Harvard Business School (HBS) . 

The primary, issue which ig addressed 
in this article is : how ccui linear pro- 
gramming be taught so that it will be an 

'effective ^planning. tool which will be 1) . 
undergjiood and 2) used ? The authors be- 
lieve that an interactive teaching method 
promotes substantially more usable and" 
longer lasting, learning than the tradi- 

;'tional lecture method, especially among 
students who wil3/ become tomorrow's gene^ 
fal managers. Often, in a Mecture-i' 

• oriented classroom^ the n625m" requires that 
the student be" attentive and carefCilly 
tramscribe the instructor's .word$ from ; 
blackboard to notebook, In-cl^sHecision 

iBteiking is rarely -required. This^^ reser- ^ 
ved for take-home problem sets and 
{usually only after hours of "cramming**) 
exams.. Often the" student forgets much of ' 
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the content of th^, course shortly after 
the exain is over. In our experience, 
this* happens less-frequently if -the-stu- 
dent ' . . 

- learns by analyzing* actual or 

. near-replicas of business situa- . 
tions in which linear programming ■ 
, has played a part, . * 

- actively participates in. class 
discussions of * the analysis, im- 
plementation, and related mana- 

, ^ gerial%^ssues, ^nd 

has easy access to an interactive, 
on-line computer package used .in 
analyzing these situations. " - - 

Since the student . is -at all time^ person- 
ally involved 'in the learning process-, 
hi&v tendency is to in teg r ait e the concepts^ 
discussed »and ^thus tc-retain them, 

' «• " " * 

The remainder of this article will 
describe' the' linear programming segment 
o^^ the required quantitative methods 

-.("Manage rial Economics) course taught in ^ 
the first y,ear of the MBA program at Har- 
vard. Section 1 v(ill give a short over-- 
view of the sct\ool's objectives studjent 
characteristics, and stricture of 'the 
first-year program. S'ection 2 will de- ' 

' tail the. -'initial; case used in this seg- 

*mer>t {tbie . durable Shermah Motors case) 
arvd the pedagogy ihvolved iti teaching the ■ 

•day-long;, introduction to linear^.-j^rogram— 
miijg. ^Sgction 3 will c^escribe tSxe next 
four' :^l^fit^ons in -which the -students are 
graduii^fS?. presented with more involved 
and fi/eal»is tic situations. Since^ an im- 

^3C5^ant ''aspect of the* learning experience^ ' 
is the' interactive lii5^ar progranming 
"Ba'ckage/i'that is eif^loyed,. the ^papaSil^- 

' 6ies of this papkage will be describM.- 
ThiS'^^eport, is -largely . descriptive in 
nature s6 that the reader can fully ' . 
a'^reciat« the unusual cdftstjraints which 
a'iris^' aict ^BS and how the ensuing diffi- 
cultieg»^x;e overcome. ' ^ \ ' ' 

1 . Qvervxew: Philosophy ana- Structure , 

The Harvard Business School MBA pro- * 
gram gives men arid women training for 
line -management positions. The intent "is 
to prodtice general i St s, not specialists, ^ 
nor^ technical staff, personnel, "'so that ■ ^ 
nether mathematics nor theory is 
stressed. For ex2ariple," the s^.mplex tyieth- 
od se "is not taught. (Graphically, 
the vertex- following concept .Remon- 
strated, but the details of the algorithm 
or the theory are not jdiscussed.J The 
emphasis 4-S"iristead on formulation of ap- 
propriate models and the subsequent anal- 
ysis of the economic and managerial 
implications of the solution results.- 
The *ratidnale £or this is cleaf?:: our 
BelJLef is thkt -general managers have, no 
need to know hov to pivot, wheMS, 
prder to. effectively use linear program- 
ining/ the. interpretation of- results is 
"essential. (in fact, we suggest that the 



reader try to' recall the last time at 
which he needed to perform, a pivot by 
hand! ) - - - - - — 

. ■ ■ ^ In the first yeat of the MbIa program, 
all students- are. enrolled iti the .same 
courses. The approximately 800 incoming.. . 
students are divided into nine sections^ ^ 
each o'f'^Vhich." will remain together in the 
same room for all th^ir 'classes' through- 
but the first year. By the sixth week, 
each section, has^ started to develop the 
tight-knit social fabric whic^ so charac- 
terizes "^he Harvard*3usiness School, first- 
year -section. -This, even more thanc^the 
well^Hnown case method, makes participa- • 
tory education p<?ssible.' The student',* 
' - "surrounded by* peers, wiidmj. for the most 
part, . he trusts and .respects ,M,is willing 
_ to go out on a limb, try -new directions ' 
and experiences. The class' size, approx- 
imately 85, insures that all points "c^f 
view will be represented and that most ' 
ii^ortant issues will 'emerge. 

Students come-from a wide range of 
geographical areas with an even broader 
variety of backgro^^nds.. and expexiences . 
This diversity al^so helps "to keep class 
discussion intere'sting.^ The "majority of 

- the class enter with at' least two yearW - 
of business experience'^ . Th,ere is no math- 
ematics prerequisite and, ' typically.,* pre- 
vious training ?.n mathematics ranges f ^tcmI 
Ph.D ' s (few) to ijone-since-tenth-grade u 
(more freqijent) . One might believe that 
this -situation might inevitably lead to - 
the poorly- trained few being lost or to 
the highly-trained becoming bored, but 
the attributes of the participatory- 
learning can prevent this. For example, 
in the first or second week of the year, 
one of the .students' wisJ:h. a higher degree 
of technical expertise will inevitably 
make a remark involving^a: complex or the- 
oretical'- concept not understood by most. 
He migfit then be asked to role-play:; one 
of the class's *'poets" will be -ca^ as 
his "boss" and ask him.to explain *'his com- 
ment. Often he tries futilely.' This 
drives home ,the lesson that the ability 
to^ explain one's technical methods to ^ 
" those/ with less mathematical training is ^ 

^ as crucial in tlie' business world as are 
those methods themselves. As, the year 
progresses, the different learning -roles 
become more clear: the poets "-^concen- 
trate on content - hew concepts and tech- 
niques? the "engineers" refine and 
abstract and learn to comm\anicate their"^ 
■knowledge. 

The core guantjitative methods course, 
consists' of five segments tau^t over a- 
six-month period: decision arialysis 
(includiJ^g the basics of probability)," 
forf^casting '(mainly multiple linear re- 
gression) ,* simulation, competitive de- 
»*cision making (el^entary game-theoretic ^ 
concepts) , and linear programming. The 



next two vrfections wilj/ describe this 
. Jlatjber ,.s e gme rrt l!_ ■ . . 

^ ^- Managerial Economics Day 

TJje introductory sessions on linear 
I>xOgraitoing, euphemistically called Mana- 
, gerial Econpmics Day, take up an entire 
day for the first-year MBA'^, during 
which interactive sessions are inter- 
spersed with discussion sessions. Our ' 
objectives ar^ three^fold: 

1} to introduce* students to the 
; . recognition .of situations in 
which linear progxWning . might 
be useful, 

2) . to indicate the basic structure 
of a linear programming model, 

. and 

3) to introduce students to the 
interpr(«'tation of linear pro-. ' 
grammin^ .solutions through 'an 

^ interactive' computer package . 

ri..r -^^eu""^^^ used " in this introductory 
day^is Sherman Motor^s*, an example adapt- 

.ed from Robert Dorfiaan's "Mathematical 
or Lineal Programming: • A Nonmathemati- 
^ cal Approach," American Econ omic Review 
December :i953. The example is ,a simple 
resource ^allocatipri problem in^^lving a 
motor manufacturing fi^^jp with two truck 
models,. moAel. 101 and ^02, to Tbe piroduced 

iising4four Jcinds of capacity: * metal 

engine- assembly, 101- assembly 

and 102 assembly. Given the ^d^t^ in the 

case., the linear programmingfSiodel below ' 

can be^ formulated. 

• * maximize 
subject, to 
metal stamping: 
,engine assei^ly: 

101- assemblyf 

102- assembly: ' 
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, where- and N^^^re, respectively, the 
•monthly production of 101 's and 102 's * 
This simple. model . is depicted graphic* 
cally in. the case -and students, with ' 
no -previous, ijitroduction, are assigned 
J. the task of deciding ■ what to do with it. 

• . . The example provides a good tran- ' 
sition from sit;iiations involving- -uacer- 
tanties but. few complex constraints, 
to those where the uncertainty is minor, . 
but the constraints are Essential. • The 
simple formulation p^its the' -easy dis- 
cussion of^basic .concepts such as deci- 
sion variables, objective function, 
feasible set, -aniJ' so on. Many students 
analyze the gi^phical formulation by / " ' 
^°"f \^rizex comparyi^ strategy 
. (in this example,^ enumeration -is easv ^ 
•Intercollegiate during Hbuse #6^107-010 



.*AlQws the .introduction- 

o£ th^ vertex-following concept. 

At this point, after a, general dis- 

m^a^^n^°^ ^^^"^^ linear program- 

ming and a more specific discussion on 
■the formulation of the Sherman Motors " 
example, the students are introduced to 

■ 5^ ^■^K^®"'^^^''^ compute, package thfey 
^tlK^^-'^^i'^^ ^"'^ sent.j:© the terminals 
for their first_set of exercises, a 
brief description of • that-~p.irogram is 
needed here. ' • . . 

• CLP*, conversational linear proqram- 
a^nn^v,^^ iS Called, Operates by alking- 
a number of directed questions ami allow- 
ing the user to sequentially request a 
series of options! it allows the user to 
input a linear programming model, to edit 
ar.i^ file the specifications, and to re- - 
-or^h^cr'' in several ways'. - The output 
^or the Sherman Motors example is given 

■ in Exhibit' I. Output sect.ion 1 gives the 
«?h! f solution; section 2 the values cf 
«Sthe slack and surplus variables; section " 

.the value of the dual variables or 
shadow prices; section. 4 the. reduced 
costs for non-basic I variables, i.e., the ' 
decrease in th^ objective which would 

^ occur If the variable was introduced 
into the ba^is at a unit level. Output 

. sections 5 and 6 give ranges, on the ob- 
jective function and.rig^it hand s^^de . 

, values for -purposes of sensitivity anal- 
y^^f 1. Section 5 gives ranges on the 
coefficients of the objec^iveo" function 
-such that, the optimal solution-^rpmains ' 
unchanged as long as -th^ coefficient • • 

sec^t^!^ -^^^^ ""y^"- Similarly, ' 

1 gives- ^ranges oft the right hand 
side valines within which the shadow 
prices are constant. , ' ' ^ 

is alv^n i^^'lKh^^ computer eltercises' 
IS given Exhibit' II. Their purpose is 

lationJ?-T' ^""^ strudent3 to^ the conver- ' 
sational linear ^rogramning - package - 'in- ' 
^ut edit, and optimize, "andT in6rl,im- 

wav th'^' P^*-"^'' an-' exglriential - ' 
way, the concept of a "shadow price " Thfi" 

asklfto'^r '^^"^ ^^'^ student" a:^e"'' 
asked to change tlie amount -^.f the avail- 
able engine .capacity from 3333 to 3334 . 
^then to-4333, then ,to 4433. The firsJ 
change causes an increase in the objec- 

to the shadow price' 
fQ:r that constraint', the second chance 
^h^^r ot 10i)0 times thfe^ • 

shadow price, and the £hird change, which 

^he'raice^n'^^' "^^"^ "'^^ outslTe"^ 
the range prescribed (see Exhibit I, out- 
put section 6V and therefore- requires a ■ 
thai%H^^?^X ""^es an increasi of Lss 
than the llOQ times th ^- shadow, pricp 
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"^^^ <2eveloped by Stephen ' 
Btadley.* For more in'foraiatlon, sej 

using the Conversational Linear Proarain- 
mihg System," ICH #9'-;L72^240 
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_which the students exp ect, e ^Most students 
when examining - the .outpu-c , "n»hich "h as not ' 



Litput ; * ^whi 

yet been interpreted for, them ^ will 
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lotc: For each cf .thes« quostions, start frO:;i 
* the basic assiLtiptions of case- • (Do 
' «ot cor'ri^ over ^idditional a^Eurrptions • 

froD one r;u?^berec question -tc the rext.) 

Find the bcst.product nix for Sheman-^tot-crs 
under th-a basic assur;ptions cf the t^.^e. 

i. Find the b<»st product nix if it is fcund 
. - ^that the st^npin^ departrrent capacity if 
< . *'-^3>S0O xcdel 102's (as in the case) b-Jt onl^ 
2, ,050 >todt>l 101 's- . ' 



^^3. Kh«t is the. best product nix if ■ engine as-^ 
sembly capacity is raised to Khat is 

aa extra unit of engine ^atssxirnbly capacity 
worth? ' ^ , 

4. ^ Whet are l/oOO. additional units" of' Xcdel 101 
'equivalent cngire assembly capacity worth? 
What* ^bout 1,100 units? j 



notice that the number listed under . shadow 

price— for- encfine assembl-y capacity- is the 

same as the .increase in the objective 
.function which they have just obsierved.** 

.After the completfion of this first 
computer exercise, there "is a return to - 
the classroom for a 45-minute class ses- 
sion in^which'the instructor answers 
questions and reinforces the shadow price 
v^_£oncept. The students . then, return to. the 
tentiinals for a second' set of exercises 
ais shown in Exhibit III, It is more com- 
plex than the first- and requires reformu- 
lation rather than simply revision of 
coefficients.. The first question intro-^^ 
duces a third product into the production 
process. It is important to note that 
this -new truck is not produced in the re- 
vised optimal solution. T^he students are 
thus" presented, again experientially 
' rather jthan directly, to .the notions of ,■ 
J- reduced cost ^nd sensitivity on the 
object-ive fxinction 'coefficients. The 
secOpd question requires a cotnplete refor* 
mulation with the addition of two new. 
' variables and a new constraint.- It pro- 
vides the frcimework for a concluding 
' ^iscussUon of the CLP output when the 
. stuHents return" for their, third class 
-..session. Exhibit IV gives a typical 

sciiedule for this introductory linear pro- 
j^gramming day. 

The student's reactions to this 
method are positive. Linear program-- 
ming is a topic which the MBA's at first 

Exhibit 1:^1 

SECc>r3 S7:r or ir.\:'.".zzzzs 

Mote: ' ror earl^jf thcdc questions, start fron 
'\ tJ^e ^^iSiC ''as&u:?pt±cns of the case. {Dry 
not carry over ^.dditionil assunpjions 
from oTjfc. nurrJacrcd jy'uestion to the ncxt.^) 

• * ■ . r 

1. Sheman .Motors is considering introducing a * 
new economy truck, to be called Model 103. 
\ The total T;iotal stanping capacity would be 

sufficient for 3/000 Kodel i03*s ::ef nonth. 
while the total onginjc assembly shop would 
be enough for r500">:odel 103's". " 7hc 103-s 
could be asserisled |n the 101 a&sen'Jbly de- ^ 
^ jpartr;ent; each 103 would require only" half 

ta5 much tine as a ICl. Each y.odcl 103 would ' 
*give a contribution of S225. 

a) JJormulate the production decision with 
^' tne three trucks as a linear.- program- 

t\inq problc:n and then solve the problem 
;* with tto^vcrify that no .Vodcl 103 's 

- ' siiould be produced. - - 

' * • » ^ 

b) Ho^ much would it cost in terns of con-' • 
tributicn if, for sone other reasons, 
laanagcrrent insisted that at least bhc 
Model 103 be made? • 

^- ■ • 

c3 How high would the contribution on each 
103^ have to be before it became attrac-^. 
tive to produce the new model? *> 

2. The engine assembly line can be put on over-^ 
\ tine. Sifp{>ose that efficiencies don't change 

and 'that 2000 units of ovortinc capacity ar« 
available. If /direct labor costs increase 
^ ' by 6;^% on overtime and if the fixed overhead 

on^ the ^ine on overtime is S<0, 000 the var- 
"7 iable overhead remaining the sarie; would, it 

V ' pay to go on overtime? 



TY?I^/^ CUsSS SCIIE:DfLE rOR M.E. DAY 



?:15-10:30 
10:45-1^1:45 
11^545-12: 30 
J2;30- 2:00 

2:C0- 2:45 
2:45- 5:30 



First Cl<iAs Session 

First Computer Sesision 

Second Cle'.s*; Session s*. 

LuiX<^h and Prepare tijOn of Second 
Exercise Set • 

Second Computer Session 

Final Class Session 



lind difficult and this interactive ap- 
proach gives them immediate feedback and 
response. Rather than &eing^ thrown to 
the. computer with no plass tiiscussion un- • 
tiZ, at .e'arliest, the next day, "they have 
.their problems cleared up immediately. 
For the most part, they are willing to 
. give up the better part of a day for ttiis 
experience, and^ the^increased level of 
.learning is noticable. 

3. Sessions Following M,E. Day 

After Managerial . Economics Day, a 
sequence of four hotir and twenty rfitiinute- 
classes, spanning a two week period, are' 
conducted pn further suspects of linear 
programming fundamenl^ls. The primary 
purpose of* tfiese classes are to reinforce 
the conce3t>ts - which were introduced during 
Managerial Economics Day and to show how 
linear programming can be implemented in 
a 'variety of decision environments. The 
pedagogical format is. as follows. We 
. be^an the first of the^ four classes Vith ' 
^ set of exercises in^ which the students 
are^ expected to successfully formulate 
^our sm^ler 'li'hear programs, run them . 
on CLP, and interpret tjie solution re- 
sults. • The stjidents- are then gradually 
introduced to. more difficult formula- ' — ' 
tions in tne remaining three days. By 
the filial class; they have become famil- 
iar with some of the basic issues involjp ' 
ing model formulation and Selection. * 
During these four sessions V number of 
pther issues arise, such as^ifiow to deal ' 
with uncertainty and the mechanism for 
pricing out the ;ion-basic variables. The 
highlights of these, four class sessions 
are described in the remainder - of this 
section. ^ • . 

The first -ses-siofi consists of simple 
formulation exercis.es — a macfhine shop 
problem, a • transportation "network., an ad- 
•.ditional market ing^pnstraint on the SJ^er- 
man Motors problein]!*and. a simple bond 
portfolip investment in which the shadow 
pripesv/ife not particularly meahingf ul * 
because- -the^constr^rnts take the fcnti of' 
ratifos. A topic of note in*^the teaching, 
of these exercises 'is the manner in which * 
the student's participate i.n teaching as 
weli as learning. Since interaction i5 a 



_c_rucial_ asjpec^^ the ..case^method.,.._thc?, 

instructor may start the discussion by 
asking a non-quantitative student to s^iow 
; how he or she. solved the first exercise. 
If there, is little difficulty with this 
aspect, another non-quantitative student 
might be asked to qualitatively describe, 
the formulation. Eventually- the' conver- 
sation gets around to discussion and de- 
f ininjg the ^various basic types of con- 
straints that can occur in a^linear 
program: \ . 

-.product (quality mix), - 

- capactiy or resource limits, and 

- supply-use constraints (flow).- 

If there had' been a severe problem 
.with the initial formulation, the follow- 

' ing pedagogical device* has proven 4isef ul . ' 
The confused student is asked to concise- 
ly and verbally ' describe any 'single ' 
constraint while the instructor tran-. 

. sdriiies the verbal* .description onto the 
blackbcpard. At this point, the instructor 
can either show how the sentence can be' • 
easily parsed into an equation or ask for 

. volunteers to do likewise.. In this mamer 
• the poets are shown- that almost anyone 
can' formulate mathematically 2^ Linear pro- 
gram, provided a precisejBescf iption of 
each restriction is developed. We are' 
careful not to ovQ2>rhelm the students^ 
with mathematical, sophistication at' the 
beginning, yet ensure that their logical 
reasoning is sound and rigorous. " 

The next class session depicts a 
simple capital investment decision -prob- 
, lem, i.e., the Mitchell Enterprises case.* 
Here the ^problem is- to formulate a linear 
-programmina model for investing in' five 
projects (A, B, C, D, E) over a foxir year 
. horizon with the objective of maximizing- 
the accumulated value of the portfolio. 
An/"opportunity rate is not provi<Sed; how- " 
^ ever, ,a 6%^ short-term bank account is ' 
•^available for any moneif which is not in- * 
^vested in a .gdven year.* Exhibit V depi.ct's 
^the cash flows per dollar'.invested ^or 
each of these projects and years. 

♦ ■ * ^ 

^ ' Exhibit V - ' - 

\ • CASH PER DOLLAR IXVESTED . 
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Year 'i375 

1975. 

19?7 
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+1.00 


V.30 


0 . 


■ 0 


-1.00 
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+1-00 
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+r.V5 


+1.'40 ■ 





with ^his infiormati^a, tfte students are 
asked to formulate ^nd evaluate a linear 
programming m^del for .this 'p roblem. " In 

♦Intercollegiate Case- Clearing House >^ 
#4-176-160 * , - ■ . . 



addition, to the case, they are given in 
_^_advance'a supplement containing the CLP 
solution to"" this'^'p^^ "In the class 

discussioa,, the following important topics 
are usually covered: 

1) a review of model formulation/ 

2) .a demonstration that the oppor- 

tunity rates can be derived from 
the optimal shadow prices,.^nd 

3) • a proof that opportunity, rates ■ 

are not necessarily constant for 
each year . t 

• Irf addition, a. brief discussion , of how to 
deal, with uncertainties via sensitivity 

. analysis takes place if time remains. 
This case has been well received because 
of its implications on tr^iditional dis- 
counting methods - a finrf's hurdle rate 
depends on the ^set of opportunities i-t \ 
has available.^ In addition, it is not a 
typ;Lcal production scheduling sit^uatioh — 

^ whioh many synonymize with linear program- 
ming mo dej.s. . . 

"On the third day the Red Brand Can- 
^ ners* case is used. This case involves 

;the production- of canned tomatoes tomato 
' juice, and tomato -paste . Information 
about fixed and vaxi'able costs, allocated 
overhead expenses, Nmarginal profit,- and 
so forth, is provided- . The s'tudents are 
» agciin asked toj;^f^5rmuiate the linear pro- 

• "^ramming model and tp defen{3* their form- 
ulation in class. This session is note- 
worthy be.cause tlje case forces students 
to extract .the c^^levanj? accotinting fig-: 

J ures ar>d to use this information within\ 
'the model .-5* It also Vividly brings out 
tile limiting assumptions of "linear pifo-\. 
grairaning, i.e., constant returns to scaj.e; 
continuo;us variables, 'non^negativity and 
so on. ^s far as pedagogy is concerned, 
the class discussion can be constructed 
to.pompare linear, programming with "back 
of the envelope-seat of the pants" ap- 
. • proach. To accopiplish .this compari'son, 

* ^here^ is an appealing;, but incorrect, 
formulation which can be proposed by the 
- instructor as the easiest way to solve 
^this scheduling problem. -A "better" stu- 

, dent will count'er by describing why the 
intaaitive approach is incorrect and 
giving the- correct formulation. Often, , 
other students are unprepared to chal- 
lenge this alternative formulation ^nd 
the class anxiety rises until the logK> 
cal fallacy i^ uncovered- The purpqs^ 
of this' subterfuge is demonstrating Jj^ 
tRe studfents^^hat they must be preg^ed 
to defend tl^Sx formulation to their ■ 
superiors ^^n ^though their. superiors 
may not understand mathematical models 

*^ or symbols, ^his important lesson is . 
<^ifficult to accomplish outside the case 
format . ' • / ' * - ' 



^Inter^llegiate Ca^se- Clearing House 
V #4-.17^-102- , • ' . ' 



in the final linear^ programming 
class session, the Okanagan Lumber Comr: 

" 'pany*"case' Ts" presente"^ "This "'case"Tn- 
volves the production of^ plywood, paneling 
from various types of lumber. The inter- 

- mediate steps in' the'^pJywot)d mill 'ar6 
described in detail. .The problem of 
scheduling and . aLli5cati«ng resources .is 
depicted as a linear progreun. - Sinc.e the 
model has approximaterj' 50 constraints 
and 75 variables, the tableau is 'provided •' 
to the students. They are^asked severar ^ 
sensitivity^analysis questions and there- 
by develdp an appreciation of the model. 
By this time, ^he students see ..that they 
can understand even a rather complex fdtm- 
-ulatiOn if given enough time t^ sift 
through' the details. Becaixse ^of its cbfn- 
plexity, there -are many ways to teach 

. this case. Perhaps the most important' 
lesson to be learned is that th^ modeling 

. process can .uncover r\ew' avenues rather 
than jiist showing how-to' travel old .paths. 
To demonstrate this idea, the instructor 
"can utilize the shadow prices in conjunc- 
tion*' with prevailin g m arket prices for/ • 
lumber^^to "invent" nSw product lines. ' In 
essence' the students learn hov to price- 
out-^ nOn-basic variables -"^ whenever a com- 
bination 'of marginal profits minus 
marginal costs is positive 'the product as 
profitable arid worth ' introducing, which' 
is equivalent to saying that the product*s 
3fe'duced' cost is positive and can thereby , 
enteiythe basis at a positive level-; This' 
case *qoncludes the linear programming seg- 
ment of tjie Managerial Economics course. 

As a final observation, we compared - 
the case approach with the *usual lecture 
method. One. of the authors taught linear 
programming to M.S.. and Ph.D. students ' ' 
from Harvard's Division of Engineering ' . 
and Applied Mathematics. This class fol- 
lowed -a more traditional lecture forsmat - ^ 
and tiie students generally had more exten- 
sive mathematical training (linear alge- 
bra, etc..) than MBA students. .The same 
case given to MBA' s oti tijfeir final exam 
vas" given to these students as a part of 
their final examination. It is interest- 
ing to observe that the non-quantitative 
:MBA' s ^examination results" compare favor- 
ably with the Ph.D. student's examina<tion 
results^.^ The ,"poets" were especially 
adept at putting the model in context with 
respectvto tjie. total investment' decision 
including^ how to deal with the underlying 
uncertainties. ^ 

4, Conclusions 

We have attempted to show in this 
article how linear programming fundamen- 
tals can be^successfully taught to a j 
diverse audience of MBA st:^'""*^ts.* ' A 
parallel situation exists ■ 
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executive nwnagement program at Harvard, 
in which highly experience managers r^- 
— ---turxi-to-acadcmia-for -a-l 3 -*reek retraining 
period. ,The linear programming aspects 
^ of this program are almost identical to 
those described in Sections 2 and 3 of 
this paper. ^ ^ ' 

^ Although we feel that oiir peda- 

gogical approach accomplished the goals ^ 
which have, been .set forth, several im- 
portant questions .remain . First , does 
the successful application of -sophisti- 
cated mathematical models require an ^ 
appreciation by the decision maker '.of' 
the solution strategies in. the model? 
An airline pilot may 'not comprehend the ^ 
^ physical laws of aerodynamics^ but he 
can still fly. Nonetheless / av, thojrough 
undeMtanding- of the. airplane can be an ^ 
advantage to the pilot whenever 'somethin/ 
goes wrong. Perhaps this is why ^ auto- ' 
mobile^ race drivers are intimately • 
familiar with the roecanical aspects of 
. . thei:]tf machines. 

We. suspect that the pertinent answer 
to this question lies in vthe resolution 
of the tradeoff between intention and 
- practicality. In other words,; since 
threre is limited time for learning, 
priorities m^st be established^ The 
chosen mix of applications and theory 
depends upon how one resolves these 
issues. ^ 

As a second question, how do busi- 
ness schools deal^with field study, the 
recent phenomenon which is occuring in 
many law schools? Here teams of students 
address real-world problems by going out 
xnto government or biisiness.. Because 
of the size of th^ mbX program at the 
Harvard Business School, it would be 
difficult to implement such a program? 
however,, this- topic will need to be 
further explored. 

Thirdly, will the dramatic improifc- 
ments in mini-computers, telecommunica- * 
tions, and sc^f -paced -learning' techniques 

\ have a significant impact on how linear ° 
programming is. taught? One of. our 
associates has developed a cassette-based 
system* for decision analysis, and simi- 
lar ideas could be easily exploited in 

» linear programming. Sa^reral business 
schools are already engaged in develop- 
ing self-paced instructional, materials. 

^ — 
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ABSTRACT 

. In" this paper the authors discuss their expe- 
riences in converting a lecture-oriented mathema^t- 
ical progrkinming course to the Self -Paced lostTuc- 
;<tiqaal (PSI) format. This is an elective course 
- for students from all the departments (Engineering, 
Science, and mnagement). We will discuss the dif- 
ferent strategies, including the development of 
conversational computer progrums, which were em- 
" ployed to implement the PSI concept invfchis"* course 
so that its educatiopal objectives are fully met. 
We will also discuss oucr experiences with the PSI 
system, its pros and cons, and the students' re- 
sponse to self -paced learning. 

* Introduction 

Undergraduate enginee;ring cxirricula have 
become much more flexible during the past half- 
dozen or»so years. Students are able to obtain 
a measure of ^ specialization in one or two areas 
v±tt3fin a specified engineering discipline; for • 
exauj^le, in t^ School of Industrial Engineering 
at Purdue the undergjraduati* students can special-- 
ize iri Operations Research, Systems Engineering,'' 
Human* Factors > >feLnag|ement, and Manufa'bturlng 
Processes. Mast of ,t2ais flexibility has been 
obtained by increasing the number of elective 
courses allowed in the curriculum without in- 
crecising the' credit hoters require^ for gradua- 
tion.' ^ ^ / 

The author 'has been teaching for the past sev- 
en years a course on Mithematical Prograimning for 
^undergraduate and graduate students. This is /an 
elective course for all industrial engineering 
students. In addition to students from other 
branches of engineering, the course attracts stu- 
dents Troffi industrial' management, agricultural 
-economics, mathctesatlcs, computer science,, and 
statistics. - Because of this, the composition of 
the class and the interests of the students vary 
widely. The course is offered three times a year 
and attracts 150-200 students. 



The basic objective* of this course is to in- 
troduce the students to mathematical modeling (in 
particular linear prograjnming models) for solving 
real-world problems. The major portion of the 
course deals w^th solution techniques for these 
models. &om past experience in teaching this 
course, i^^lias been observed that the non-engineer- 
ing s^dents prefer to see more emphasis on linear 
programming theory and computational methods. In 
contrast, the engineering and business students * 
want more emphasis on case studies describing nu-* 
merous applications of linear programming. They 
• even express varied interests in the linear priJ* 
iiii'affiaing topl^ to be discussed in class- Por in- 
stance, the mathematics students want the inclusion 
of. advanced topics like game theory, and decomposi- 
tion methods. The computer science students like 
to see the computer Implementation of linear pro- . 
gramming algorithms. The civil engineers show 
more interes.t in topics like network analysis, 
transportation problems, and critical path methods. 
This poses et^rmoUs problems in structurJhg and 
teaching this course. In previous years, a middle- 
of-the-rroad approach^was taken, so that it would 
hopefully satisfy most of the .students ' interests. 

Use of Personalized Self-Paced Formats 

" Traditidnally all studtents enrolled in a spe- 
cific course progress through the.- material defined ^ 
by the syllabus e^t a given rate. Concepts pre- s 
sente4*i° ^ textbook are supplemented with regu- 
larly ^scheduled lectures by the course instructor,- 
'homework problems are turned it; at the proper time, 
lab experiments are performed during scheduled 
hours> and hour tests are 'taken in class size 
groups when the instructor s.chedules them. For--' 
tunately this is not a rut in which the innovative 
teacher mast remain.. • Through the years we have 
seen the growth in xise of concepts of' indiviual- 
ized, self -paced instruction and open laboratories 
(see References [1], [2], [3]». *nd [If]). The 
American Society for Engineering Education has been 
concerned" with effective college teaching for a 
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dumber of years and usually (ievotes thec March icsue 
of Engineering Kducatlon each year to articles con- 
^cerned with innovative methods of teaching in 

--•-cnrTinetrrine^-CTirrlcTlla-^ 

[5] reports various -innovative teaching: metnods 
used in industrial engineering course^ across the 
country. However, we often find that the najor dif- 
ference between the self-paced verij.j&p of^ a cour::;^ 
and its lecture counterpart is- merely the self- ' 
, pacing; there is still a strict list of topics to 

be covered. Variatiorr within the listed course 
^content is only offered to the student through the 
vehicle of writing a terra paper on. a special topic 
.of- his interest, not included in the syllabus. 

- • The author tof. this paper felt the need to 
expand the amount of flexibility,. ol'fered to a stu-'V 
dent enrolled fn the ^mathematical programming 
course beyond that which is described ±t\ the pre- 
ceding paragraphs. Because of the diversified - 
interests found among the students, it was felt 
that a new Personalized Self-Paced Instructional 
(PSl)^ system could best serve the students '"needs. 

- Jhe new PSI system developed by the primry author 
f7] combines the traditionalT lectures, self--paced 
and mastery learning to provide maximum flexibili- 
ty. To obtain any degree of^ course topic flexi- 
bility it is necessary to evaluate each course in 
terms .of its educational objectives. What 'is the 
terminal behavior *we desire of a student when he , 
completes this course? Then, in order to make 
additional, optional material available to students 
enrolled in this course, it is necessary to deter- 
mine the minimum, necessary requirements from the 
original course content. In 'this way spe(?if ic 
topics are made required content of this course. 
After this is completed,* the additional (optional) 
■material is made available to the student in the 
form of elective modules. 

. Thus, in the new syste^, the subject matter' 
for this course" is divided into some basic -units, - 
and some 'Optional units. Class lectures 'are given 
for the basic 'units only. The students elect ^ne * ^ 
optional units they want to learn according to . 
their interests, and prepare^'for^ them on their owti 
time (off class hours)* For this reason, one- 
third of- the scheduled class hours are cancelled 
to provide time for independent study. 

BASIC UNITS : The basic units are so chosen that 
they provide the students a minimal amount of 
knowledge in the fundamentals; of linear program- 
ming. These include: ■ ' * 



Unit 1 - Formulation of Linear Programs : To 
construct linear programming models of real world 
problems . 

■ / . " ■ ' 

Unit 2 - The Simplex Method : To learn how 
systems of linear equations are solved j to under-''' 
stand the T^asic mathematical principles underlying 
the simplex algorithm; to use the tableau form of 
the simplex method to solve small problems; and to 
use. the computer code for solving large linear* "■ 
programs. ■- ^ . 

Ilnit 3 - Duality Theory and its Aptfllcations : 
Symmetric i^nd asymmetric duals, [ econotrfic iaterpre- 
"^tion, dualiiy ^thdorems and applications, shadow 
prices, and l^he 'dual simplex method. 
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Unit U - Sensitivity Analysis : Variation of 
.cost coerricients, changing Rris constants, changing 
_ constraint^ matrix, .a«3ding new -constraints_atid para- 
metric proZ^ramminp; with respect to cost .and RHS 
vectors . ' ■ * ' 

OPTIONAL UNITS : The optional units . are select uo 
provide diversification, and meet the varied t.- r: 
special interests of the students. For instanc 
during the summers of '7i+, '75, and '76, when tr.- 
PSI system was tried for this'coui^e, the follo^*i;;c: 
optional units were offered to the students: 

Unit 3 - Project Networks and PEKT/CPM 

Unit 6 - Transportation and Assignment Prob- 



lems' 



Unit 7 - Variants of the Simplex Method 
Uni t j - Nonlinear 'Optimization - 
Unit_^ - Bounded Variable Linear Program 
Unit^lO - Bimatrix Games . ' p 



Unit 11 



Integer Prograjiiming 



In addition, an individual project is always 
made available as an optional unit, ^or each of 
the units (basic and optional) the students are 
provided with study guM. - Hussell [SJ, calls 
these "modular materials" a nr. -^roviaes an excel- 
lent guide to the ~eEi.:n, ::el-_- - : Ion, utilization, 
and evaluation of zher.e stu'ly .:ides. Basically, 
th'e study guides coi^a... '.h. - 0 jectives . of the unit, 
topical oGtline, reading materials, and sample prob- 
-lems. Examinations or. various units are given ' ' 
throughout the s erne's tcr. "Thoxigh excellence is re- 
quired to pass a unit, th- students can repeat the 
unit examinations any nuirbrr of times without - ' 
penalty. For a miniziBl r .j-ing grade in the course, 
•the students have, to complete all the basic units. 
Assirnment :.:-aer grades is a function of the 
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they. pass. 



ComT'iter Aids ~: the PSI System 

Fro::: the initial experience in teaching thi^ • 
cc..r::c in PSI format, it '*as found that consider- 
able time was spent with -he- students', by both the 
professor and the teaching assistants', in helping 
tr.em learn the optional units for which there'iwere 
,no formal lectures. Tnis increased the manpower 
needs of this c .ltsc. It was felt that this could 
be reduced to a'great extent by using the computer 
as a teaching aid for self-paced learning. The 
computer-aided teachi.-.g can h^lp them learn the ' , 
various linear programming techniques and their 
applications on their own time^(on/off class 
hours), and at their own pace. 

■ With the help of a teaching grant fVom\the 
Purdue j^rents^' Association, interactive ci^puter 
programs' have be^n developed in such a way that 
. they will illustrate and test the students' under- 
standing of the si^bject matter. .The students solve' 
problems illustrating various linear programming 
algorithms in a Conversational mode by answering* 
a series of questions. The programs are designed 
so that the computer does all the tine constiming 

/ . ■ . 
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arithmetic calculations, while the ^'thinkinf?/* is 
done by the students. The student esaentially di- 
, rects the computer to calculate whatever quanti- * 
. ties' are "needed" ■ f or "s olving * the "^problem "by " a'~given " 
algorithm-- Based- on these, he/5he instructs the 
computer to. proceed step* by htep in the required 
manner to arrive at^the r>ptlmal oolution to the 
problem. .We have provided in the program a- yefi- ^^ 
fication routine so that alL the s-teps" -and in-' 
structions given by the stddents are checked by ^• 
the computer- for correctness'. 

The arithmetic calculations in linear pro-' 
gramming techrriques generally consume more than 
' 90^ of .the tbtal time in solving; a problem, felnce, 
"this time* is eliminated by the computer, the stu- 
dents are able to use their study *^ime more.'ef-' 
fectively. They .can solve a variety of problems 
'to test their understanding. A series of illus- 
-"trgitive problems emphasizing various concepts and 
practical' difficulties encountered in solving ^ 
linear programs are also available to the students- 

* Thvis a student gets a better' and a more complete 
exposure, to the subject' matter its'eir. At present 
the interactive programs are used to^ reinforce^ 
me"tKodology and to interpret results. The' uhit 
examinations do- not utilise these interactive ^ . ; 

• programs. . • \ • ■ 

Description crC the Interactive Computer ^ Program 

Conversational Features * ' 

' The .program completely conversational- *A^1 
input arfd output -is handle^ via remote terminals. 
The system always directs -the user .to the 'next, step 
of the simplex- algorithm by asking a question' re- 
lated to *some aspect of the algorithm. At all ■ 
times, the user's' response is 'checked for correct- 
ness.* *If .the user should respond incorrectly t& ^ 
one of the system's queri^s^ an immediate feed- ^ 
y back ^ill be sent to the user with an expdanation 
of why the response is incorrect and how/ to go 
^bout" finding the correct answer. 

Error Detection for Inappropriate Student Responses 

Each student response to a system question is 
checked for correctness. If the • student' s answgr 
is correct, the next question is etsked by the sys- 
tem. However^ if the response is not correct, 
feedback explaining th^ error is given at the ter- 
minal. In some cases, the same question is re- 
asked and the student is given- further opportunity 
to respond correctly. In other instances, the 
system supplies the correct answer anc? then pro- 
ceeds toSlhe next question. . - ' 

Error checking is accomplished by;* havlLng the 
system solve" the give'ny^inear program "step-W-step 
; as the user solves the^ s^e problem. For ins^nce, 
before asking the user to supply the index of xfie 
pivot row in the simplex algorithm, the system will 
have determined that information. Then the ^user 
will be asked to supply the same- After the user 
enters the response, it is a simple matter to 
check the response against the result already de- * 
termined by the system. If the user's resgonse i^ 
correct, the next qt>estion is pr-fnted on the ter-" 
, minal* If the response Is incorrect, a message to 
that effect is printed with a brief explanation of 



why the response is not correct.^ 

- - For instance, on line A (pg. 5 ) of the enclosed 
">xample""in ~the"Appendix7 tW" student' is" ask€d,~""13 " 
THE PROBLEM IN STAITOARD FORM?". 'In this caaeil the 
problem is ndt in standard form. But the student 
has responded, "YES". Thus, the feedback given is 
''INCORRECT RESPONSE-^ THE PROBLEM IS NOT IN 'STAN* 
DARD FORM, NOT ALL- OF Tlfe CONSTRAINTS ARE. EQ^ALI- 
TIES". Then the- next quJpstion is generated*.'* \ 

. User Assistance v 

In: addition to the brief explanations provid- 
ed when the user gives an incorrect response, 
anotjfier^ form of explanatory assistance is offered 
by the syst^ero. At several points, the -student is 
.asked to name the next step of" the algorithm. ' The 
possible responses .and their meanings are: " , 

4 ^ " 



RELPROFIT, -'.print the, relative profit 



.vector 



RATIOS -; 



PIVOT 



calculate ratios of right- 
hand sides divided by thtf ' 
corresponding entry of the 
' ^ivot column 

perforra^;a pivot ^operation 



In any instance where one of the above responses is 
"th'e correct one, the 'user- may type^'HELP" if he is- 
'.uncertain of wjiich choice is 'the correct one. Af- 
ter "HELP" is typed, a message indicating vhich 'is 
the correct choice and why it is ^correct will" be 
provided* possible extension j/ould be to allow 
the us€:c to ask for "HELP!* at fny* point during the 
execution of- the program. ^ This should be" included. . 
in. the next version of the '-system. 

An exajaple of the use of this command is given 
-in line,B (pg. 5) of the accompanying example. The 
question asked is, **WHA'tf IS THE NEXT, S^TEP?". In'"-- 
this case, the correct, response would be "RELPROF- 
IT", since the user mxist look at the relative prof- 
it vector. The, user has. type'd "HELP.'* and thus re- 
ceives an appropriate explanation- 



Inplementation Description 

Progralmming _Lan^;^ge Requirements 

The system was programmed in FORTRAN- TV on 
the CDC 6500. A language with better character 
string handling facilities would have been a better' 
choice- -However, the constraining factor was the 
interactive field length restriction on the Purdue 
system.- Interactive* jobs are limited to a maximum 
field length of 21K. ^e ob"Jett" code g^erated by 
FOR*rRAN was able to fit into this space and stitl 
allow some room for extensions.. On an IBM system 
with time-sharing, APL would be a. better choici^. 

Algorithmic^es_cription 

, At present three mathematical programming 
algorithms have been prqgrammed in the interactive 
system. These include a primal simplex and a dual 
simplex algorithm to solve linear programs, and a 
branch" and bound algorithm for solving integer 
linea:r prpgramming problems. Each program's logic 
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.is based^ on the description of that algorithm g£ven^ 
r • .in .the text 4)y >Ri±lli ps, '^Rfcivindran, and Solberg .. 
y\ [6}. S'tuitenia,%e'askfed input;, problems of their 
own*r*oosing.{ijriiis -allqw^^^ flexibility *" 

fli the instructional p^^oblens dio^en rather than 
limiting the' selectiaVvfeo j^t a few. The ibstruc- 
■ to^.cax Also suggest some problems which demon-* 
s^trate the bets ics df'the algorithm. .After uast^- 
ing the ^basics, tb'e students can-'experiment with 
^, problems, *of tjieir own afioosing- 

■ . * 

. Concluding - Rexoarks / • ^ * . ^ 

- \\ '^Th^.'iSI -system is clarrently being, well re- * 

ceived by the students. The atrfchors have ^valu- 
' ated the cohrses by several measures': cdinparison 

with test' results in other, years, student election 
^. of more advanced courses in these areas, student. 
• acceptance -as indicated -on writl^en course cri- 
tiques, etc. .In all cases,' the self-paced method 
of instruction wa^^udged'in general to be superior 
■for this particular course- (See Table 1 for sttf- 
^ 'sdent^ comments*^ 0 ' This is not to say that th^ usual ' 
/• problems of some students frilling -behind schedule* 

^did ci^t-'exlst,'^ because they didr^ Effort is ex- 
^ pended during 'the semester to help ;student motiva- 
tion. Also, the PS I system consumes xaore faculty 
time initially in the planning of the units, and 
preparation of- study guides and tests. 

Considerable time is still being spent on an 
individual basis with some students in helping them 
learn the optional units for which there are no 
formal lectures. To offset this the author has ob- 
tained another research grant to support the prep- 
aration of video cassette tapes for the optional 
units. The School- of Industrial Engineering has 
recently acquired' a TV monitor and a cassette 
playback unit. These video tape*s can be played 
on these -units by the students at their own time. 
We hope to have these tapes preoared by the end of 
the yeiar for students' use. 



Acknowledgements 



-The support for developing the interactive 
computer programs came, from "an innovative teach-* 
ing- grant awarded' to Professor Ravindran from ' 
the Purdue Parents' Association. Preparation of 
the Video cassette tapes" foi* TV .viewing .will be 
supported by a faculty, grant awarded to Professor 
Havindran for Instructional Development and In- 
novation. * 



• Reference? • J . ' , /P^" '* ^ 

[1] .31ocK, J- H.^,. Master^'- teaming ^ Hol^t^ Rine-- • 
. * . , hartj and Winston, ' . ' - . * 

[2] Dell, Individuaii'zing Instfuctibn » , 

'Science Research, Ass6ciates , Palo Alto*^ ^^pali^ 
.-fornia, 1972. * ■ . ' '"T^- • \ - \ i 

[3] Esbensen, T. , Working^W Individaailized ';in- 
struction-\.Fearon, Palo Alto, California*,^ 

.1955;^ ' ' . ■ • ■ ^ ■ - ■ V " ^ 

• ' ' i/. - ■ ■ ' 

Keller, F. -3.,. "Good^Bye Teacher . . . "> Journal 
• of Applied Behavior ^Analysis , No^'l, Spring 
1965. . ■ ■ 

[5] 'MooSie, C. "Report bn a Survey of Jnnova- 

■ ^, tive Teaching MettyDds Used in Industrial Ea- 
".'gineering Education"^ Research Memorandum 
K"o. 73-8, -School of Industrial Engineering; 
^Purdue University, West Lafayette, Indiana, 
July 1973- - " " - 

\Sl Phillips, -D-.; T.,^Ar' Ravindran, "^nd J. Solberg, 
Operations Research: Principles & Practice" , 
John Wiley & Sons, New York, I976. 

[7] Ravindran, A- and C. lI Moodie', "Increased 
Course Flexibility Through . Self -Paced In- 
struction",^ Proceedings of the Frontiers in 
Education Conference (ASEE Atlanta, Georgia, 
October 1975, (pp. ^0-^3). 

[8] Russell, Z< D. , Modular Instruction , Burgess 
Publishing Co., Minneapolis, Minnesota, 197^- 



APPENDIX 



The solution to the following^jAnear program 
by the interactive simplex algorithm. is shown, as 
an illustration: J • 

Maximize 2 = + + Ux^ ' ^ 

Subject to + + s 100 

° ■ ^ ---^ ' ' lOx^ + i^X2 + 5>y « 6oO 

2x^ + + 6xj s 300 

Xj^, Xg, i 0 
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LUJSAH* Programming self- paced i n st r ucI'ion 



• ENTER THE^KUMBER OF COKSTRAITfe FOLLOWED BY 
' A.BBAUK FOLLOWED BY THE* NUMBER OF- VARIABLES 

^ ■ r 

= 55- * ■ ■ . . \ - ' ' 



.ENTER THE LINEAR PROGRAM . - 
IS THE OBJECTIVE FUNCTION TO BE MINIMIZED OR-- 

: MAX * ^ 

ENTER THE C VECTOR * • ' , 

: 10 6 it ^ ■ ' • 

ENTER TK^ A MATEIX^ ROWWJSE 
111 

2^ 6 ^ ■ . ^ ■ 

ENTER THE B VECTOR ' ' . * 

100 600 500 ^ = . 



ENTER.. THE CONSTRAINT • TYPES : 
' <.FOR LESS THAN OR EQUAL TO, - 
> FOR GREATER THAN OR EQUAL TO, 
= FOR EQUALITY CONSTRAINTS 

: < < < 

(A) .IS THE PROBLEM IN STANDARD FORM? 

■ : YES. 

INCORRECT RESPONSE- - 

THE EROBLEM IS NOT IN STANDARD FORM 

NOT Ali OF THE CONSTRAINTS ARE EQUALITIES 



CHECK THE CONSTRAINTS* TO SEE IF THERE AJffi ANY 

NEGATIVE RIGHT-HAND SIDES. ALL SUCH CON- 
• STRAINTS NEED TO BE MULTIPLIED BY -1. 
' ■ WHICH CONSTRAINTS NEED TO BE MULTIPLIED BY 

-1? ^ - * ' . . ' ^ 

TYPE O..(2SR0) IF NONE f 

0 " ■ • •. , 

FOR EACH CONSTRAINT, TYPE A' + 1, ' -1, OR 0 
DEPENDING ON WHET5iEi^ A SLACK VARIABLE IS 
NE EDED, A SURPLUS VARIABLE IS N^ED, OR 
NEITHER ONE IS NEEDED ■ 

1 1 ' ■ 

THE. SLAC< AND SORPLUS VARIABLES HAVE BEEN . 
INSERTED ■ V ' - ^ 



ABE ARTIFICIAL VARIABLES , NEED ^J)? 
: YES 



.INCORRECT RESPONSE - \ 

^ NO ARTIFICIAL' VARIABLES ARE NEEDED BECAUSE 
THERE ESfcCSTS AN^DTTTIAL SEASIBLE BASIS 



V,^THE .CURRENT TABLEAU IS : " 

■■ CCSST ^10 6 :\ % ■ 0 . - 0 

■ XI X2 X5 XU X5 X6 v'-B* 

BASIS ^" . ■ - - - 

Xh 1. 00 1. 00 1.00. 1.00 • .00 .00100*00 

• -X5 -lO.poVii.oo 5.00 ■ .00 1.00 .00600.00 



x6 



2.00 e.oo 6.00 .00 .00 1.0 500.00" J, 



"what are THE'. VALUES OF THE BASIC VARIABLE? * 
IK THE ORDER. IN WHICH THEY APPEAR IN THE " 

. "TASiEAu ' ' ■ ; • ■ - ' ' 

: 100 600 500 ■ / • / 

WHAT IS C(B), THS vector of COSIL COEFFICIENTS ' 
OF THE Bi^JSIC VARIABLES IN CORRESPONDING ORDER? 

* 0 0 0 \ 

THE VALUE OF THE OBJECTIVE FUNCTION = .00 

(B) what IS THE NEXT STEP? 

CHOOSE FROM PIVOT, RATIOS, OR RELPROFIT 
IF YOUJDONT KNOW, TYPE HELP 

: HELP 



THE NEXT^STEP OF THE SIMPLEX ALGORITHM TO ■ 
LOOK AT THE RELATIVE PROFIT VECTOR AND 
DETERMINE WHETHER THE CURRENT BASIC FEASIBLE ' 
SOLUTION IS OPTIMAL. DETER2OTE THE' PIVOT ■ ' • 
COLUMN AND HENCE THE VARIABLE WHICH ENTERS 
THE BASIS BY CHOOSING THE VARIABLE WITH THS 
LARGEST RELATIVE PROFIT COEFFICIENT. IN 
ORDER TO SEE THE RELATIVE PROFIT VECTOR, TYPE 
RELPROFIT. 



WHAT IS THS NEXT ST£P? 

CHOOSE FROM PIVOT, RATIOS, OR RELPROFIT 
IF YOU DONT KNOW, TYPE HELP 



: RELPROFIT 

THE RELATIVE H^OFIT VECTOR IS: 

lb. 00 6.00 i^.oo ■ .00 .00 .00 

» IS THE TABLEAU OPTIMAL? ■ 



YES ^ , . ^ 

, INCORRECT RESPONSE - 
FOI? A .MAXIMIZATION PROBLEM, ;;TKE TABLEAU IS 
OPTIMAL WHEN ALL OKE RELATl^fe PROFIT COEF- 
FICIENTS ARE NON- POSITIVE 7 -■ 

\ / 
SELECT THE PIVOT COLUSIN 



•WHAT IS THS NEXT STEP? 

CHOOSE FROM PIVOT, RATIOS, OR RELPROFIT 
IF YOU DONT KNOW, TYPE HELP 



ERIC 





OKB NEXT ^£5, OP THE SIMPIEQ -ALGORITHM • 
THE PXVOT EOW ARD HENCE 
' VALUABLE WHICH LSAV BS T HE BASIS USE THE 
[J^TIO RULE TO DE^gERMTNE THE BASIC 
TO lEAVE THE.BASIS«TO-CAIiCUlATE. 
RATIOS TYPE RATIOS . / 



: RATIOS ' 



.•FOR ^EACH ROW, TYPE A 1 IF A RATIO OTEDS TO 
BE GAXTOLATED .OTHSBWISE TKE A 0 " ^ 

: cJ 0 0 V 

7:&'jfflSR)KSE FOR ROW 1- IS lUCORRECT 

' THE- RESPONSE FOR ROW 2 'IS UJCOHRECT • 

THE RESPONSE FOR ROV;-^ IS INCORRECT 

THE CORRECT RESPONSE IS: 1 11 

THE RATIOS ARE 

ROW NO* • RATIO 



1. 

2 

3 



100*00 
60,00 

150.00 



CAN A PIVOT OPERATION BE PERFORMED? . 
:'N0 

INCpRRECT RESPONSE - ^ 

A PIVOT OPERATION CAN BE ^3^RMED SINCE .AT 
LEAST ONE RATIO HAS BEEN CALCULATED 



SELECT THE PIVOT . BOl^^ 

f 5;; . " ' ^ ' ' 

INCORRECT RESPONSE. - " 
'THE ROW WITH THE SMALLEST .RATIO IS THE PIVOT 

ROW . •• ■ ■■ - . ■ ■ ^ y*'" 

SELECT THE PIVOT ROW 

■ - / 

: 2 " . ■ . 

'WHAT IS THE NEXT STEP? 

CHOOSE FROM PIVOT, RATIOS, OR RELEROFIT * 
IF YOU DONT KNOW, TYPE HELP 



:^ -PIVOT 

THEJ CURRENT TABLEAU IS: 



4 • a J' 0" 

X3.. JCl+.. ' X5' 



- 0 ^.• 

x6 



B 



,HELP 



NOW THAT THE PIVOT ELEMENT HAS BEEN 
DETERMINED^ "A PIVOT OPERATION -CAN BE 
PERFORMED. TYPE PIVCT "TO PERFORM THE 
PIVOT OffiRATION . 



WHAT IS THE NEXT STEP?.* 

CHOOSE FROM PIVOT/ RATIOS, 'OR RELPROFIT 

IF YOU^DONT itNOW, TYPE HELP 



COST. 10- -6 
' • XI X2- 
BASIS • ' * ■ • ^ '" , . ■ 

xh <bo .6a' .50 ' i.oo. '.-Ao ' .00 . to. 00 
■xi..ivOo"-".;i+& .;.50 .Ob v/.io ■ .00, *5o«oo- 
x6 .00 ■•i.2d^ .5.0a . .oo'v-.2o 1:00 iBo'.ob 



y- 



WHAT ARE '4HE. VALUES OF THE BASIC VARIABLES 

in the"6hder"in.which ohey appear in the. 

TABLEAU 
: i*0 60 180 

WHAT IS C(B),* THE VECTOR OF COST COEFFICIENTS 
OF >THE BASIC VARJA5LES IN CORRESPONDING 'ORDER? 



: 0 10 0 ■' ^ . - . 

THE VAUJE OF' THE OBJECTIVE FUNCTION 



600.00. 



'WHAT IS' THE NEXT SIIJK ^" 
^CHOOSE FROM PIVOT, "'llATIOS, OR RELPROFIT- • • 
IF you, DONT KNOW, TYPE HELP 

- ^ . 

^ : RELPROFIT - " 

THE REIATIVE PROFIT VECTOR IS: 

.00 ^'^^ -i.oa .00 -1.00 .00 

'. IS OHE TABLEAU OPtlMAL? ^ 
: NO ' 

*- SELECT THE PIVOT COLUMN ■ - " , 

: 2 ■ ' ; ■ 

WHAT IS OHE NEXT STEP? ■■ 
■ CHOOSE FROM PIVOT, 'RATIOS, OR RELPROFIT 
- IF YOU DONT KNOW, TYP& HELP 

: RATIOS ... . - • " 

• FOR EACH ROW, TYPE A 1 IF A RATIO iffeSDS TO BE 
CALCULATED OTHERWISE. TYPE AO ' . " 

: 1 1 1 

THE RATIOS ARE 

ROW NO.' ■ RATIO. * ■ ^ 



. 1 
2 
3 



66.67 
150 loo 
150.00 



,CAN A PIVOT OPERATION BE PERFORMED? - 

: YES ^- ■ 

. SELECT THE PIVOT ROW • '/ ' ^ 

; .1 , ■ " -f- '\- 

WHAJ is THE NEXT"STEP? 

CHOOSE FROM PIVOT, RATIOS, OR RELPROFIT 
IF YOU DONT KNOW, TYPE HELP. ' 
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'303 



PIVOT 

'raE"^CUHHENT"TflBIEAU IS: 
COST- 



0 



X5* 



- 0 

x6 



B 



basis,^: 

X2 ...00 1-00* .88 1.67^ -.17 .066-67 
XI .' 1-00 .00 ."17 -.67 -.17^ -0 55.55 

x6 jKoa -00 ^♦'.oo -2.00 -Job 1.0100.00 
whal^^e the .values" of the basic variables 

IN CTE ORDEa IN which 3EEy APPEAR IN THE 
TABLEAU ■ ^ . . ' 

* : . - . ^ 

: 66.67 .35.53/106.00 \ ; ' 

WHAT IS"-T^), THE VECTOR' OF COSi COEFFICIEKTS 
^ OF THE BASlte^AR'lABLES IN CORRESPONDING ORDER? 

: 6 10 ^'^ ,J - 

THE VALUE OF IHE OBJECTIVE FUNCTION = 735.55 
WHA^ is* THE NEXT STEP? 

CHOOSE FROM PIVOT, ^RATIOS, OR RELPROFIT « ^ 
IF YOU BONT KNOWp. TYPE HELP 

: RELPROFIT ' ' 



-THE REIATI^ PROFIT VECTOR IS: 

^.00 .00' ,-^2.67- -5.55 - -.67 

IS THE TABLEAU oiPTimL?', 
: YES 



4OO 



DO YOU WANT TO SEGIN A NEW PROBLEM? 

:*.N0 - • 

+++LOG 

TOB L024 11.1;4.51. 10/p7/75. 
estimated' SESSION COST $ .17 
PLEASE TURN OFF TERMINAL- TNX. 



Table- 1. Results^of VStuden^3urvey 



TTo* oT Students EnraHsd . - • ^ 

.Ko. StudenffcB Res ponded - " ' 

Reaction^to the PSI System ; >- ■ 

a. Favorable initial response (semester beginning) 

b. ^Favor&ble final response (s'epester end) . 

^ '> " 

c*- .Helped mfe to acHieve desired goals . ' 

.' ' s;n ■, --^-v.-iv : .'. ■ / 

d. _ Able to plan'-jiy 'stti&ies- better- ' . 

e. Helped me to get a better .grade 

f . , Would have preferred tradit;^onal f omat - 
g'-. Learned more under PSI -Z- 

h. Worsted harder under "PSI V • 

' , ' '' [^ ' ' * - . 

i VVv, Worked ' less under * 



Summer, 
■ ^ 25 

23'' 



Yes No neither 

-13 ■ 3 



18 
19 

•.IT 



5" 
5 



id •; 8 

a 13 



"5^ 

6 

■:i' 
i 

5 
0 
"6 
2 

2^ 



Summer 1975 
- 36 



Yes /No- Neither 



.33 
35 



r 



31 
22 

.1 

21; 



-1 
5 
7 
32 
=; 



3 

2 1 



18 15 
11 21 



0 
7 

.3 
7 

5- 



Summer 1976 ' 
39 • 



Yes Ko geitbgr 
~2F 9 2 



32 
30 
■29 



6 
8 
9 



r 



-27 10 

6 30 

2h 11 

11; 21 

ll; 21; 
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IMPORTANCE OF MODELLING 
FOR' INTERPRETATION OF ■ 
LINEAR PROGRAMMING PROBLEMS 



Lepn^rd*-W. Swanson 
Nor.thwestern University 



Abstract 



The construction of the rSatbejiiatical inodel 
for a linear programming probrfem- requires extreme 
care in. order that^it ^e effective- If one u^es 
as few variables as possible, he may find that' it. 
is efficient in computation time but in^ef feet ive ■ 
for sensitivity ffi^aJ^ysis^lS^ This paper Tises a re--, 
latlvely uncomplicated "example to^ show the way in T 
which proper modelling enables one to ;^>ytract im- 
portant analyses not obtainable throug^Ca simpler 
model* The appi^bach has been success ^fiy used_ 
in teaching linear programming » particularly fo?""-^ 
expljaining the concept of duality- fhe. same 
methods can be highly effective in^^nagerial 
applications- * • 



_ Introduction 
* ■ . " 

It is often thought that one is being very 
efficient if he solves a Linear Programming Prob- 
lem^ by using the fewest possible number of vari*- 
ables.- If one is interested only in the solution 



and not in any kind of sensit^ivity" analysis > this 
may be true^ but if one is interested in the 
effect of a variation of the parameters of the 
problem, many more 'details may be required to en.- 
hance the 'analysis and interpretation. 

Scheduling Problem 

lifti- pifder. to consider thfrf benefits of effi- 
cient 'Wxieliirig, .consider, the production schedul- 
ing problem .stoown in Klgure 1 (no claim is made 
. for. the original it;y of the example)- 

i ' , ■ . ' " • ■ 

A .plant makes two products , A and B,, which^ 
are " routed througly, four . processing centers * 1, 2, 
3, and 4 as showrTby the solid lines in ^ the en- ; - 
closed diagram. If there is spare capacity in 
center "3, it is possible to route product through 
3 (dotted line) instead of goixig through* 2 twice^ l 
but this is m'ore expensive » . ■ • " ' \^ ■ 

Given the information in Table X and Table 2^^ 
how should production be scheduled,. so as to- .maxi- 
mize profit? (By. production schedule is meant 



PRODUCTION SCHEDULING 




B 



Center 1 ^ 



Centte 2 



Center 3 



Center 4' . 



I 
1 

^ \ 
I 

... .'i 

r 

^? 
J 

^ ^ ^'"^V ^ ■ 



.J 



(alternate) 



' Figure i ^ 
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the specification of Che folilowilig three .imou^Ls: 

"! > •• ^ \ ' -v : ^ • . . ■ 

(1)* The daily axaaunt of raw materi^if i«ed ■ 
for A, regiilar routei^ • \v 

/ ^ I ^ 

J ^' "(2). The daily aniDuiit of raw matex^al usea'^- " 

^ * r f or'A, bptlonal routa> . » *' * 

.* ^^ ■ . • . ;^ 

(3) The daily amount of raw material used 
■ - . ; .. .for B. 

Assume that sufficient storag^e capacity^is avail-, 
able at no additional cost.) 

..." ■ . . 



' Centers 1 and 4 run up to' 16 hours a- day; • 

/ > . centers- 2 and 3. run up to 12. hours a day/ A fi*\al 

^estr^tioti is furnished by shipping facilities/ 
•j^ vhidh d'imit the daily output of A and B to a tfotai 
ot 2500 gallons- . ' ' ■ ' ^ ' - ^ 

^ r . C^bijsider first $hc formulation of .the" prob- 
lem in a form in which' as few variables las pos- . - 
isiblfe are used. Therefore . v , 



Let^ * the number of gallons o£ ini>ut ,of 
.A, which^jJ^Xo follow -.the re"feular 
t ' processing route ^ 



A - 



Center 



201st pass) 
4 

2 (2nd pass) 
3 



1 

B H 3 



V 



Table ^1 






Running 


Inputs, 




cost 


gals- 




per hr- 


per hour 


recovery 


S 


300 


90 


.150 


450 


95 


. 200 / 


250" 


85 . 


180 ' 


400 


80 


220 


350 


75 ' 


250 


500- 




300 


480 




250 


400 


80 


240 





■ : / 


9 

Table 2 




•Product 


^ Raw 
Material 

•"cost 
per ^al. 


„ Sales Jprice 
per finished 
Ral. 


• 

Maximum dally 
sales, gal. of 
finished product 


A 

B. 


5 
6 


20 

" 18 


1700 

- 1500 — 5^ 


5 










' ■ it, - 







X2 ""the number of galJjO^s o^ input of 
- * A, "v^ich is to foltow the alter- 

nate processing route ' '\ 

" the number of gallons of input .of 
- ' ' B, which has only one route. * 

Figure 2 is a representation, of the problem 
- incorporatiiig these variables and also the infor- 
mation from Tables 1 and 2- 

'On each path through a particular center, 
there are two numbers ; the left hand number .is the 
number, of gallons per hour that can be processed 
and the right hand number is the cost per hour for 
processing.- ' . 

Mathematrical' Statemenf^ of" the Problem 



Pind X 



1 * ' 



2 0 



Such that 



capacities 



5xj^ + 5x2-+ 3x3 < -'24,000 
13,74075x^ + 7.2x2 - ^3,200 
34.8.84x2 + 31.5x3 2"Gl,'6Q0 
6.84x^ + 6.84x2 .+ 3.825x3 £ 32,000 



^ J3Q0 



$150 



400 



$220j 



.5814X, 



Center 2 j pec^ 

450 $200 •^^^'^l-?250 Slgaj 



300 $150 
Center I 



500 



$300 



.9x3 



-$2flOJ .855x^ 



■ % 



r" 
x 





■$25cr 


Center 


3 


350 


$250 





.765X3 



250/ .$180 
Center 4 



400 



$240 



> 726751 



^1 



.72675k^ 



4> 
I 
1 
f 

• I 

.612x « 



.5450625x^ .T^ ^ . 



(alternate) 



Figiirc 2 



-i-r. 




" ^ ^ r- ' ' ^ 
Sales^ -T^SSWxj; f • 



545062Sx2^^ -1700 



'p;5814x ■ .5450625x:,> .612x^.5 2500 . 
Capacity 1_. ^ • »1. ^ . - . , 2' 3 . 

and such that ^ ' • • - , 

■ • 

V , 4.>Li62875x^ + 3.866543x2 + 3.48825x3 

.* • ' , 

is a maxi^tiiffl* . ; J.. ' *" 

• . . -Optimal- Solution ^ 

In this fQpn of the problem, ^one can get. the 
optimum solution carrying out the production 
schedule. The solution reveials that 

■ 1.- 2923.98 Salons follow the regular' route 
A but none follow the altematfe roiitV^ 

,2. 1307.19 gallons follow route B. • , 

3. The prof it^s $18,399.59. 

? 4., The actual -output of either A or B is not 
revealed .directly although tt can be ob- 
< talned by' additional simple- computation. 

5. Through the dual solutions one sees that- 
Che sales restriction on A is a limita- 
tion' on profit and that Dock« Capacity is 
also a limitation .on profit. The per 

' un£t gains made by relaxing these con- 
. - straint.s aJC^e given by the dual solutions. 

6. Nothing in the solution is directly re- 
vealed concerning the effect of - chang- 
,ing either the rates of gallon througfi- . 
put for each of the centers nor the , • 
hourly costs of processing in each of v 
the centers. 



7* A -Study jof the effect of changing the co- 
5,fficients of the objective function is 
'not very revealing since the coefficients 
in this function are .a conglomeration of / 
a Aunber of. incomes and costs. ^ ^ 

' ■ ■ ■> : ^ . - ■-. ' ■ 

■ This^ -f arm of the.,^godel 'seems t-o be^relatively 
neat and^'iDne might even pride himself in^the byeV- - 
ity of ■. the format .thinking that restricting the 
problem to.-thre^ variables and seven constraints ^ . 
,is- a minor triumph.' , 

'It would appear that a manager interested in 
p|.roduclng* the most effective schedule vduld require - 
^a sensitivity analysis . which would study the effects 
' of changes in processing rates and processing costs 
fpr ,each of the centers* Accordingly, in^. what -fol- 
lows I develop a model of the ^ame' problem which 
will. enab^« the manager to Xearn a vast amount more 
about the C7pera:c ion and which would enable him to 
do a betteii' job -of- effective management. 

■ ■ V , 

' Alternate Statement of the Problem > 
^ - — ^- 

As. shown in Figure 3, and xg are nsed to 
^ represent t^ie raw material inputs for A and B res- 
pectively.. These variables are modified by re- a 
^ covery rates and new definitions are made so that 
variables XX to ^iesignate various inputs and 
outputs' in the processing sj^ream. From Figure- 
it can be .seen" that a solution . f or xy gives the 
final output' of A^, regular; a solution for X3 gives 
the final out^.ut of A, alternate; and a solution 
for X12 Sives 'the final output of B. Furthermore^' . 
the definition*of these, vari-abies serye as con- 
straints which involve the 'rec orrery rates, The'de- 
■ flnitlbn of' variables x^^' through X2q serve as' con- 
straints which define the hourly processing rates 
of the ^:enters.* ^The recorery and hourly, processing 
definitions -and corresponding constralxits^ are given 
below. , ' . 



A » ■ 



{ 



I 3C 



13 300 
(a$1507hr 



-hrs 



n6 ' 400 
(a$220/hr. 



-hrs. 



^14" 450 



hrs. 



(a$200/hr. 



x^ 



18 500 



^ri 



<a$300/hr. 



r ■ 
I 

4k '. 



=^19- 



"10 



480^ 
(a$250/hr. 



hrs 



'11. 



^15 250 
(a$180/hr 



■hrs. 



^2G"400 



@$240^hr. 



A (regular) 



5 ' / 




(a$250/hr. 



¥ 



"8 . 

— \J- — — —-♦-I A {alternate) 

I 

... * . ^ 



Fl] 
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Recovery definitions: 



- 1 X J^"" ^ 9x, 




Xg - ..75xg 

XjQ- .90Xg 



A. 



. - Wh/sti written 
^ • ,as ^constraints : 



.90: 



- 0 



Xg "^'.'TSxg ^» 0 



T.2 .11 / 


x,^ -'♦SDx ■»'0 


« - ■ 

Center 
Proceissing times: 


* When written 
as constraints: 


^^j3-x,/300 


x^ -350x^3-0 ^ 




^ -'^50x^^ - 0 


x^5-- X3'/250 _ . 


X3. -250x^3- - 0. 


. x^g - Xj/AOO • - 


'■^Xg -<400Xj^"g " O.- 


• ''i? ■ - ^ 


¥g -350kj.7 - 0 ^. 




Xg -SOOx^g'- 0- 






X2g-Xjj/400 





We add the constraint which shows the balance 
for A regular and A alternate route as 



X. " Xc + X^ 

4 3 6 



X£ wie then add the seven constraints of our 
snsaller ssodel in terms of tl;e above variables 
we ^ave a model with twenty-four constraints 
and twenty variables. The problem is then 
stated as follows : 



Find Xj^ to x^q 5 0 



such that 



Recovery 
Definitions 



.Hourly 
Processii^g"* 
■Rat^s 



Balance^ 

Center 
Capacities 



Sales 

Constraints 



Dock 
Capacity 







s 


.90x, « 0 " 

1 / 

• 


X- 


- • 953t. - 0 - 


* x/ 
4 


- •asx, - 0 




- .BOXc « 0 . 


0 


- .15^^ -0 ; 


^10 


- . 90x^ « Q- 


* • 

11 


- .SSx,' - 0 
10 




- %80x,T - 0 ^ 


■ x', 

; 1 


- 300Xj^^ " 0 


x« 


- 450x,, « 0 
14 


X. 


- 250Xj^2 « 0" 


x_ 

5, 


- 400x,^ » 0 
lo 


X-r 


- 3^50x;, » 0 

^ 17i ' 


9 . 


- 500x,^ - 0 
18^ 


X, 

10 


- 480x^J; - 0 
19 . 


^11 


- 400x«^ --mO 




• Xc - X- " 0 

0 0 


13 


+ X • i 16 

18 ^ 


■ X- / 

14 


+ X, > . ^ 12 
16 ** 


X. - 

17 


+ x,« •< 12 

19 


X, c 


+ X20 i 16 ■ 


•."7 


+ Xg & 1700 




< 1 <;nar 



X- + Xo + X, 



2500 



7 *8 ^ *12* 
Such that 

-5xj^ + 20xy + 20x3 - 6x^ + 18xj^2 " 150x^3 - 200xj^^ 
- 180x^ - 220x^g - 250x^^ - 300x^g - 25bxj^^ - 240x2^ 
is' a MAXIMUM. ■ . ^ • 



7 
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- Schedttllng Parameters . 

The actua), parameters for r this problem in either 
form are':. .'"'■*.'•■•'■ 



1. 
2. 

3. 

4. 
5. 
6. 



Sales Litni&s on A 

Dock Capacity - 

Hourly Capacities^ 
3, and 4* 



eor centers 1, 2, 



Cost of Raw Material Input A and- B 



'Income frorn^ Sales 'of A and 
Centcr ,1 ^ • . " ^ 



for A 



,Jlate of processing (Gals/hout) 
Cost "of processing ($/hour) 
' Same^as above for 3 

Center 2 — . . 

Rate of processing (GaJ.s/h6ur)l f qj- 



- „Cost of processing ($/hour) 
Same as aSove 

Center 3 

Kate of processing. (Gals /hour) 

Cost of processing ($/hour) 

Same as above for A alternate 
-route 

9. Center 4 \ 

Rate of processing (Gals /hour) 
Cost of processing ($/hour) 
Same as above for B 



^st ypass. 

'for A 
\2nd pa's is 



I 



for B 



for A 



Advantages of the Second Model 

■ When using the short form of the' problem, we 
"are able to make a sensitivity study for it«ms 1, 
-2, and 3 only. In the larger model, we are able 
to analyze all items on the list [I]* 

Sensitivity on ^Recovery Rates* " - 

In order to see the advantages of the Second 
Model, consider the effeqt, of changing the re- 
covery rate for material '.A goiiig thrcHigh center 
1, The recovery rate for this process is ini- 
tially 90Z. It is, rather obviops that increas- 
ing the recovery rate will iiiq>rove the profit if 
nothing else. is changed, but what are the effects 
quantitatively? ). 

Consider the def initios equation' 
and its corresponding constraint ^ 



The dual solution corresponding to this constraint 
is . .. ■ .'■ . / 

, * + 6.11X11 



This -is interpreted to mean that 
were " written 



if the const! 



raint 



the ins tant:an ecus fate "pf change in profit is 
6.11111 for a unit increase in b_ . 
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^ Since the recovery rate- is the coefficient of 
1/we are really trying to do a sensitivity study 
v'olving a non-linearity but good approximations 
can be made. Since 

X2 - .9Xj^ + bj^ 

it can be seen that increasing b^ makes ft pos- 
sible to gejc the same output Xj with, less^nput Xj^,- 
i.e. an increase in recovery rate; h-^ is measured 
in gallons' and the rate of increase in profit is 
in dollars per gallon.^ ' • v 

If b]^ is increased one gallon, it will result 
in X]^ being reduced by h-^/.S or 1.1111 bj^^ provid- 
ing X2 and the recovery rate remain fixed. This 
can be seen since 

- x^ - .9(x^ - L) + - -9x^ 

.9x^ - .9a + b^ - - .9x^ 



Change in x^^ ■ ^ • ~g ■ 1.111 b^^ 
Therefore the savings will be. approximately 



l.illl i - 1.1111 {5 + - 1-111(5.5) 



6illll 



where the $5. is the p6r unit savings in raw mate- 
rial input and -150/300 isu per unit savings in pro- 
cessing costs i which checks with the dual solution. 

In order to translate, this to a change of IZ 
in the recovery rate, a.good approximation for the 
effects, for small values'-at least,, is given by 
changing b^ by an amount equal to 17. of x^^, since 

.91xj^*'-- .9xj^ +.bj^ 



leads to having 



b- » .Olx- 

1 > 1 



Using the constraint' 



X2 ■ .9x^ + b^ 



and allowing h-^ to change hy an amount equal to 
,01x1 equivalent to using the constraint 



x^ « .91Xj^ 



o 1 0 



c 



2924.0 
2631.6 



To illustrate: * ' , 

, oir solution for our^ 

--problem :W.q1<^s 

>. > 

Vhetber we use a change in bj^ or a change in re- 
covci^ r^'te of 90%, we prodiice ?631.6 as the out- 
put of center 1. - A reduction of IZ in input of 
29.240 substjltuted for yields, 

2631.6 - .-^^ • 29.240 « 0 

or X, - ^891.51 • 

processed at 90% recovery rate. 

If ve use the constraint * ' 



then 



- . 9 Ixj^ ■ 
2631.6 



.91 



2'891.87 



a difference of processing of .36 gallons @$6.11 
or approximately $2.19. 

Thus if we use changes in bj^ at 17, of X]^ and 
use the dual value for the constraint, the expected 
improvement in profit is : . 

(.01) (2924) (6.1111) - 178.69 . 

' The new, profit would then be 

18,339-59 -H 178.69 - 18,518.28 " 

A solution of the problem at 91% recovery rate 
yields a profit of 

,$18,516.32 

.The error is small but importantly the changes in 
profit are in the right direction and the manager- 
. ial implications of a change in recove'ry rate . are 
of the correct orders of magnitude. 

In a similar fashion all other recovery rate 
changes can be assessed by using the dual solu- 
tions- " It is only necessary to change the input 
to any center by 1%, then multiply by its corres- 
ponding dual value to get an estimate of the 
effect of a change of 1% in absolute value of any 
recovery rate. 

• If one wished; to find a more exact effect of, 
tbe change of recovery rate, it must be done by 
'finding the effect ,of changing the proper coeffi- ■ 
cient in a constraint- For the example we have 
studied above this amounts to changing a coeffi- 
cient for the variable X]^., which is a basic vari- 
able. The model presents the necessary informa- 
tion for this study, but . it is much "more detailed. . 



Seniit 



► Sensitivity on Processing Costs 

''; In order tq study the effects, of processing 
rates,, ' we examine the dual solutions 'Correspond- 
ing to; constraints 9j thro^ugh 16. For example'. 



if the f*ate of processing for center 1 were in- 
creased from 300 301 gallons per hour, the dual 
solution indicates an improvement of 50c in profit 

- For .any-:Qf^the above cases it is necessary , to 
study the ranging -of t^e right hand side to make 
sure that there are no problems ..associated- with 
degeneAcy. ' ' . 

Another part of the analysis would assess, the 
effects of changing processing costs f or. eac'h. of 
the centers. This can be dope directly by a study 
of the ranging' of the objective function for vari- 
'ables . ' , \ : ' * 

^13 ^^^^"S^ ^20 

The effects of .changes in raw material costs 
are given by looking. at variables 

^ and x^ 

and the effects of changes in selling price are 
. given by looking at variables 

x^. Xg^and ^^,2' 

It should be evident that the more detailed 
model provides much more managerial information. - 

> . ' • ■ - ' 

Effects on the Computer . 

The^^maller model resulted in three variables 
an<l„seven ^constraints and for the Computer Program 
used [2] at Northwestern University's Computer 
Center required .01 seconds of computing time* 
The . larger model resulted in^44 variables and 24 
constraints, and required. . 1940 seconds of comput- 
ing time, . . . t , 

• ; • ■ . J 
The program used had capabilities of sensi- 
tivity analysis. When use was made of Ranging 
both the Right Hand Side and the Coefficients of 
the Objective* Function the total computing times 
were .652 seconds -and 2.-673 seconds respect ively. 

Memory requirements were, obviously greater 
for t;he larger model but were no threat to the 
solution of the problem. 

The increase in both storage requirements and 
computing times w^re not insignificant but it i« 
-clear they do not impose diff iculties ' which can 
not be overcome. Should the modelling of a prob- . 
lem along the lines illustrated in this paper lead 
to excessive demands on either memory or computing 
*time, the problem can be partitioned so -as to -re- 
lieve the, excessive demands without diminishing 
the advantages discussed in the paper, although 
more than one computer run might 'have to be made. 

In; this example satisfactory reductions can' 
be made by eliminating the definitions . and con- 
straints, in terms of the remaining variables. 
This reduces the problem to one of 28 variables 
and 16 constraints and allows one to make, the sen- 
sitivity analysis on the recovery parameters. 
k '■ !r 

One can '.by similar means eliminate -other por- 
tions of the- problem and ft h us concent"rate on .one . 
*or two sections at a time. . " • 
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3.11 



The summary computer output or each of the 
problems discussed' in the p^er are given in the 
Appendix. 



Large Probleio 
Suaffliary of Results 



Conclusions 



The basic conclusion that can be reached Is 
that substantial galn^can be made in facility for 
interpretation by ppo^ejf modelling of the problem.' 
These gains are made^^ar^ cost of increased memory 
• requirements., for the computer and increased comput- 
ing time. .. 

Although consideration has been given to es- 
tablishing rtiles for the way in *which the problem 
should be modelled for ^uch efficiency, no com- . ' 
plete set has yet been established. .It remains 
somewhat of an art or perhaps a great amount pf 
foresight. In the examples used» an increase in 
both the number of variables' and the number of con- 
straints led, Co these efficiencies in interpreta- 
tion but such is not always the case^ There are 
other cases where the gains *are made by reducing 
both. the number of variables and number of con- 
straints. ... ^ ' ' 

Complete computer outputs are not included 
y because of the space required.. These are availab'le 
however. ' ' 



Appendix 
Small Problem 
Summary of Results 



Var 
No 

J* 


•Var 

Name 


Row 
No 


Status 


Activity Level 


tunity 
Cost 


1 


^1 




- B 


2923.9766082 




2 






NB 




.5516015 


. 3 






5 


1307.18^5425 




4 


Slack 


1 


B 


545S..5483316 




5 


Slack 


2 


B • 


. 3022.36842X1 




6 


Slack 


3 


B 


160423.5294118 




7, 


Slack 


A 


B 


7opo:^)o6oooo 




8 


Slack 


5 

■ «h 


NB 




2.4060028 


9, 


Slack 


6 


B 


700.0000000 




10 ' 


Slack 


7 


NB 




5. "6997549. 



MaaClnajm "Value -of the 

^Objective Function 



18339.591933 



. Var Var 


Row 




' Activity 


Ofporxunity 


No Name 




. Status 


. Level 


Cost 


1 


i 






2923.9766082 


■ — * 


2 


4L 




B 


2631.5789474 


■ — — 


3 


Xo 




B 


2500.0000000 


— — 


4 


- X* 
. 'I- 




B 


2125.0000000 




5 






B 


2125.0000000 




6 






B 


o.ooooobo 


• — • 


. 7 


7 




^ B 


1700.0000000 




\ 8 . 


o 




B ' 


0.0000000 




9 


Xrt 




B 


1307.1895425 


— 


10 


Xi e\ 




B 


1176.4705882 




11 


Xi 1 




B 


1000.0000000 


— — 


12 


Xi o 




B 


800.0000000 




\ 13 


* i J 




B 


9.7465887 




14 


Xi/ 

14 




B 


5,8479532 


— — 


15 


X-t c 

15 




B 


10.0000000 




16 


- 16 




B 


5.3125000 


— ^ — 


17, 


Xi <7 

17 




NB 




265.6492411 


18^ 


Xi o 

lo 




■^B 


2.6143791 




19 






B 


2.4509804 


— — 


20 






B 


2.5000000 




21 


Art If 


1 


NB 




6*1111111 


22 


Artif 


2 


NB 




6.9005848 


23 


Art if 


3 


NB 




8.9653939 


24 


;Artif . 




NB - 




11.8942423 


25 


Artif . 


5 


NB 




11.8942423 


26 


Artif. 


D 


NB 




7.3333333 


27 


Artif 


7 


NB 




9.2401961 


• 28 


Artif 


o 


NB 




12.3002451 


29 


Artif 


9 


NB 




.5000000 


30 


Artif 


10 


NB 




.4444444 


'31 


Artif 


11 


NB 




. 7200000 


. 32 


Artif 


12 


NB 




\ 5500000 






13 


NB 


— — 


-.0447121 


34 


Artif 


"14 


. NB 


^ — — 


.6000000 


35 


Artif 


15 


NB 


— 


\ .52083^3 


36 


Artif 


16 


NB 


— 


• .6000000 ,^ 


37 


Artif 


17 


NB 




-8.9653939^ r 


38 


Slack 


18 


B 


3. 6390322 






Slack 


19 


B 


.8395468 




40 


Slack 


20 


B 


9.5490196 


— — . ^ 


41 


Slack 


21 


B 


3.5000000 




42 


Slack 


22 


NB 




2.4060028 


43 


Slack 


23 


B . , . 


700.0000000 ' 


* 


44 


Slack 


24 


NB 




5.6997549 



Maxinniffl Value of Objective Function ■ 18339.59193^3"^-. 
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3NIS?ACTIVE 0CMPUTE3^ - codes' POR MA3HEMATICAL PKXSmirNG SDUCATTCN 



. Robert P. Davis* and Janes W. Qirissis 

•D^artaent of Industrial Engineering and Operaticns Research 
Virginia Poiyteciinic Institute and State Dhiversity 
Bladcsburg, Virginia j 



ABSTRflCT 

■'^^ P^^er describes the role^ of. interactive 

ocii5Xiter programs in inatheraatical progranmiM' 
education^ - Th ree different categories of inter- 
acti ve p rogragns are descrijDed and a^^aropriate 
■ envxraiuaits are suggested in v*iic± eacii shotild 
•yield i±)B greatest utility in allowing a stxadent 
to enhance his corputatiooal eaqperience with, and' 
ODno^tual understanding ^of, an algorithm. An 
exanpl^ exeooticai for a progran r^aresentative of 
e^iof these categories is given for illustrative 
purposes. . *- 

INTRO EXJ Lri'I C N 

_ are essentially three problero solving ' 

metftods enplqyed in gaining ccnputational e^qserl- 
enoe with ..nathanatical progranndng algorithns' : . 

^ (1) Manual ootputatioti. 

(2) Execution of existing "canned" progrcins 
(e.g. , batch-oriented routines) . 

(3) QcecutiGn of interactive codes for 
.exis t i n g algaritinns (e.g.,^ data ii^xit 

i . dedsion input solution output) . 

R it h e r manual oonpitaticn or execution of existing 
-batchr-oriented routines is. the netiiod most fre* 
quently €nplcyed. 

Manual cooputation is useful in ^ 

applicatian esg^e ri e n ce to small prcblenis/bpt« 
even in such cases can be,,jqdite -time oonsundxig, 
In additibn, it should.be noted tiiatt minor ^ 
matfaemattical errors may occur yieldihg erroneous 
results even thcxi^ the odnputationai nethcdol- " 
ogy is correct.- Far liiis reason manual ccnpu- 
tation freqtaently degeni^cates to mere nurriber 
m f m i pilat ioD with an abandocinent of all attaipts 
at co n ceptual interpretation. At the other end 
of thie specUxiQ, the execution of batcii-cariented 
routines can result in little'or to understanding 
of the stnirture of an algorLthni being enplcyed to 
cijtain a solution, tte method favored' ty the 
authors is one of developing and inpleraenting 
interactive codes for existing algorithms. This . 
25pro£jch is^ a^inittodly, an atUa iiJ L at reaching a - 
ocqprcmise between manual mel±)ods and batc^ 
oriented routines. Interactive corputing cap. be 
a very effective medium for reinforcing the 



learning process in mathematical programning 
education.:' Further, its inplementation can 
significanitly reduce the time required for the - 
student to gain ocnputational experience with • 
existing atlgorithms. 

^ ; DISCTSSICN 

Interactive canputer cdgortthms can be 
separated .'into three broad categories: 
... (1) 'Initial data entry. 

(2) ; Intennediate iaiteracticn (limited 
/ decision inputs) ' , ' 

(3) / Intimate interaction in fhe algorithmic 
; process! 

^ Interactive programs requiring- only initial, input 
data can be quite useful in a problem solving 
errvironmsnt vihere time^is a factor and the student 
* is" already familiar with the algorithm's operation. 
Ihe major utility fron sudi progr ams in their 
accessibility and c^pacity-to provide results which 
can:be used for interpretation of toc^I charact- 
eristics and relevance. However, such -algorithms 
are not useful in reinf coxing, the solution/iSetit- 
odology they CTploy. One exaiple of such an'"' ^ 
■ algorithm is an interacti^ routine for solving 
li T TPar progranndng prcbleros. With this algorithm, 
the student -enters 1±ib necessary prcblem data 
(effectiveness, ooiefficients, iipufc-output matrix 
and requirements vector) and the program returns a- 
sfalution. . The^ solution can then be used to .exa- 
mine TTr ri e l relevance or can provide a beginning 
" for conducting postoptimality analysis. Exarqple 
1 illustrates an execution sequenoe«for such an 
.algorithm (2). " 

^Algorithms which eitplcy limited interaction 
during the solution process are typicadly those 
vshiA require the student to enter: values for 
decisions variables/ request or abort sensitivity 
analyses, or indicate whether cptimality has-been 
adiieved. As anyillustration^ofc.such algorithcs; 
EScanple 2 ^qws an execution sequence for a 
dynamic progrcsriaing atlgorithm. OMs algorithm is 
for seiictl systens"with. linear returns function 
and - linear state transitions in a single 
variable ^t each stage. Again, such algorithms 
do not fully reinforce soliition mptfaodology, but 
they can .support an swarenesS" of an algorithm's 
^decision process as well as illustrate inherent * • 
characteristics of the models to whidi they ^ly. 

Finally, there are 'those pro gra m s which 
require an intimate interaction by the student in 
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the algorLthm's process. ' With such routines the 
stuctent nust provide the decision infonnatican 
necessary $ar the algoocithm to. progress. In this 
wspff the algorithm' s structure is en^iiasized and 
a xtore complete as^jareness of its. cpeation is 
iequired. Further/ an qpportunity for reinforcing 
.Aroonbeptual understanding and interpretation 
of the algoritian are provided. To illustrate 
'such .prograras, Qcarrple 3 shews an execution, 
sequence for an' interactive linear progranming 
aOgorithm which ijot only requires that initial 
prdslaa data be given> but further that each 
pivotiiig step.. in tiic interactive process be 
defined, and optimality and feasibility be identi- 
fied. - ^ 

V With such interactive pcogr a ns the major 
burdei? of mathematical manipulation is absorbed 
■ by t^ie to^. The student, is required to ipate 
kiey decfsions throughout the ailgorithm and is 
provided Ht »^ to e9C£Enine the oonsequenoes of these 
decisicns. It should be noted that the stodent 
is given an opportunity to observe the^ advantages 
of machine oonpitation in algorithmic operation; 
and further, interactive oodes are easily 
structured to allow tte student to observe,- in 
•ttie code itself ,.. the ocnputaticnal; building blocks 
vihich cgrprise ^ 'algorithm he enpL^sC ^ 

Interactive programs of -this last jcactegory 
can be furfeer extended . to provide' error detect- 
ion mechanisns. ttese e rr o r detections tate, one 
of two forms, firsts there are* those which 
indicate an incorrect decision but do not identify 
what should have been the correct respooose. The 
other not only indicates an error^ hqt also, pro- 
vides tiie' correct response (for exanple, in an 
interactive LP code, a chedc could be^'made cn 
vtether the miniiirum non-negative theta value was 
selected in identifying the pivot row) . . It is the 
opinicn of the authors that erroir checking 
nechanisms of -tius latter category are inapprqpSciate 
since th^ eliminate an injxaa^tant aspect of the 
learning e&^jerienoe — that of/identi^dng and 
correcting erionec«s decd^ons made in e^jplying 
solution ne^iodology. In fact, we question the 
use of ary form of error detection since they do 
not permit the student tp cfcser^^ the consequences . 
of an erroneous decision (with i±e LP exanple 
suggested above, a non-positive element in lise 
solution basis) » 

Uiis last" point brings us to an ijiportant 
question which rr^^^vp^ to. "die mind of the instructor 
\iio seeks to inplement interactive coipating in 
a teaching/learning activit^r — "What is the 
^^xtxpriate level (category) . of interacticn for 
the activity at hand?". For this, we hacve no 
absolute response* ^ general, we can state 
that tSe'^propriate category of interaction is 
a function of the student's familiarity with an 
algorithm and the purpose of applyixig the program. 
If the student is in the process of learning 
the structure of an algorithm, the last of these 
categories is the most atprcparLate- If he is 
familiar with the methodology and seeks a quidc 
solution to a particular model to examine its 
relevance or desires to initialize a soliatim to 
conduct sensitivity analysis , then the first 
category should suffices It remains .for a aan- 
prehensxve investigation to provide definitive 
guidance to the instructor in assisting him to 
identify an appropriate code .for his ^ecif ic 
activity. What most be r ecog n ized is that there , 



exists more than one level o^- interacticn to . 
•vduch such programs can be brcu^t and thatt an 
appr opriate level for one activity may not be 
dp p r c y rlate for another. .. - - 

^ ",■■'*. " . •■. ' 

OONCXUSICN ' 

Interactive cowputer cod es can be etployed 
with utility to instruction in mathematical • ' . 
progrartining education. "Jhese codes can be, 
divided- into three basic categories with each 
representing a different level (or intimaic^) of 
interaction, ahe ^ropriate category for a,, 
particular teaching/learning activity is not 
definitively established but is postulated to 
be'^a function of the student's background arid 
the purpose for which fee algorithm is being 
eqplqyed. 'As the purMse for applying the 
algaritian tends to begone of reinforcing the 
student's oooprehension of a Solution methodology 
and cmoeptual interpretation, the inti m acy of 
interaction manifested in the code dicwlxi increase. 
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EXAMPLE 1 



EXAMPLE FftOM SECTION 3.8 



THE ORIGINAL COEFF XCIEIiTS OF. THE CONSTRAIHTS 

• ^ 

CnOE 0 -■>. <0R" CONSTRAINT 
COnH|l «*> >aR> CONSTRAIFIX 
■ C00E 2 »-> - CONSTRAINT 



t COOE CONSTANT .AU^l) . At 1,21 AII,3> A(U4I AU,5) A(Cr6} All.TJ A 

^•00 • - - 

2.00 
1*00 

\ . ■ . ^ 

THE COEFFICIENTS IN THE pRIClNAL OBJECTIVE FUNCTION TQ 8E MINIHUEO ARE 
-6.00 ^4.00 ^ 



0 10.00 
0^ 2^.00 

1 3»00 



2.00 
3.00 
1.00 



BASIC SOLUTION I , • ' 

XBI n- XI 30.00 

. XRI 2)- X^l 5}» 24.00 

XBI 3)w XI 61- 3#00 

CURRENT VALUE OF TKE OBJECTIVE FUNCTION IS -0.30000000E*04 

BASIC SOLUTION 2 , . " 

XBC n- XI 4)- 24.00 

XBI 2»» Xt 5)« 15.00 

XBt 3lw XI ll« 3.00 

/^CURRENT VALUE OF THE OBJECTtVE FUNCTION IS 0. 18000000E>02 
BASIC SOLUTION 3 

XRI n« x( i>- 14.00 

XBI 2)- XI 3>- ' 5.00 * 

XBI 3)- XI D- 8.00 

CURRENT VALUE OF THE OBJECTIVE FUNCTION IS 0.48000000E*02 



THE LAST BASIC FEASIBLE SOLUTION IS OPTIMAL 
OPTIMAL VALUE OF THC ORIGINAL OBJECTIVE FUNCTION IS 



-48.00 
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TYPICAt STAGf OESCRIPTIOH: 



$<ace Decision 
d(J)<s(l)> • IC(l)*=s(l) 
0<-i;(t)<-l I 



Input State : ^ : ^ Output State 
. >:' . ' :*-s(l-l)-A*s(l)4>B«d(I}<s(l )> — > 

Stase RetMrns 
^ r(i)-C»sCl)*0*d(l)<s(l)> ^ 

Cumulative Returns 
\ fCi)-r(?)*f(l-l)<sCl-l)> 

Jf the problem Is to Maximize enter a "1" for L; otherwise, enter a "0" to Minimize. 

■ ■ . ' • . • - 

Enter the state transformation parameters: A,B as requested. 

A , ■ • 

. . ' . ' ^ 

Enter the returns function parameters: C^D as requested. 

s ^- ■- ^ . ■ ^ 

D - ^ - - ^ 

^.2" - ^ 
Enter 'the number of sta^esCN}^ up to a maximum of 6.. 

. »» ■ * 

Stases to cor X 

■ jV ' ■ 

Enter a value for IC,C 1) which will maxlmlzei^ ^ * 
-f( D- 0.5000*sC: |) ♦ C -0.2000)*KC 1J*SC 1) ' • . . 

KCl) 



Staces to toY 2 - 

Enter a val2ke -for K( 2} which will maxlml ze: 
f( 2)» 0.85€0*s( 2) C O.OOOO'KC 23*sC 2) 
IC(2) \ . 

Stases to got .5 v 

Enter a value, for ICC 3) which will maxlmlzet* 

fC 3)» 1.0950»s( 3> ^ ( O.HiOO>*ltC 3)»s( 3) 
K(5) 
1 



Staces to co: k 

Enter a value for ICC U) which will maximize: 

f< k)m 1.36JfS»sC.U.) ♦ C 0.29«»0)»»:C k>s( *) 
ICC4) 

1 c 



DECISION SUMMARY 



Stases to so -pedslon 

1 0.00- 

2 0,00 . 
- 3 1.00 

. k f 1.00 



TOTAL RETURN • 1.6515 > C *=r 
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EXAMPLE 2 . (cont.) 



Enter th« value of s( d) as requested. ' / 

■J 

- SYSTEM PERFORMANCE 



x.oo 



1.10 

>z * * > 



. y 



••••• 

r" . 0.30 



1*10 



K- 1 
• • « • 

3 . 



0.33 



0 



1.21^: ^ : 0.85^. 



0.60 



K- 0 



o.ss 



0.59 



TOTAL RETURN 



1.66 



This termTnates the procedure* 
*• 111* XEQ •*STOP". 



O.kl 
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DO' YOU KNOW HOW TO USE THIS PROGRAM? ^ ' 

? N < . ' 

THIS IS AN INTERACTIVE LINEAR PROGRAMMING RDDTINE 

YOU MUST HAVE YOUk PROBLEM FORMOLATED AS A MAXIMIZATION 

PROBLEM, 

THIS PROGRAM WILL ACCEPT >,< OR - CONSTRAINTS, 

THE TYPE OV CONSTRAINT mST BE INDICATED TO THE PROGRAM, 

THE INDICAtdRS ARE: L EOR G FOR AND E FOR 

ALL >- CONSTRAINTS ARE CONVERTED TO CONSTRAINTS^ ^ 

ARE YOU READY TO USE THIS PROGRAM TO OBTAIN A SOLUTION? 

? Y ■ 

ENTER THE NUMBER OF CONSTRAINTS FOLLOWED BY THE 
.NtJMBER OF STRUCTURAL VARIABLES, SEPARATE THE NUMBERS 
■ WITH A COMMA (,), ^ 
3,4 

ENTER THE COEFFICIENTS OF THE OBJECTIVE FUNCTION 
C(1),"?;^,„.C(N) . ^ 

? 3,4,5,1 



rENTER THE INDICATOR FOR THE TYPE OF CONSTRAINT, 

L FOR <-, G FOR. >-, E FOR ON O^IE LINE 
FOLLOWED BY THE LEFT-HAND SIDE. CONSTRAINT COEFFICIENTS 
ONE ROW AT ^.TIME:A(1,1), ,,. ,A(1,N), ONE PER LIKE, 
? L ■ ^. , . . 

? 2,1,3,1 

? L ^ ' 

,? 1,2,4,2. 

?.L ■ , 

7 3,2,1,1 ^ ' 

ENTER THE VECTOR OF RIGHT-HAND SIDE COEFFICIENTS 

Ba),^..,^,B(M), 

? 18,26,30 

CURRENT BASIC. SOLUTION: 

ROW .'VARIABLE VALUE- 



■OBJECTIVE FUNCTION - 0 
IS THIS SOLUTION FEASIBLE? 
? Y 

NON-BASIC • 2CJ)-C(J) 

VARIABLE VALUE 

1 -3 

2 - -4. 

3 -5 

4 ; -1 ' 



1 . ' 5 

2 : - 6 

3 ^ ^ 7. 



18 
26 
30 




V 



Y 
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EXAMPIE 3 {cento 



IS TOE SOLUTION OPTIMAL ? 
? K 

EUTER THE NUMBER OF THE N0N-3ASIC VARIABLE 
. TOU WANT TO BRING INTO THE SOLCTION. « 

? 3 . 

'BASIC . . PIVOT . R-H-S.^ 

VARIABLE . COLUMN VALUE 

6 ■ - X^-^ ' 26 . 

_ 7; • ' 1 30 

ENTER THE NUMBER 03? THE BASIC VARIABLE 
10 LEAVE THE SOLUTION. 

■? 5 • . • ' ■ 

NON-BASIC ZCJ)-C(J) / 

VAKCABLE VALUE 
.1 .333333 
,2 * - -2,33333 

5 1-66667 
4 .66666/ 

IS THE SOLUTION OPTIHAL ? 
? N 

ENTER THE NUMBER OF THE NON-BASIC VARIABLE 
TOU WANT TO BRING INTO THE SOLUTION, 

>? 2 

BASIC PIVOT 
VARIABLE COLUMN 
3 -333333 

6 .666667 
.7 1-66667 
ENTER THE NUMBER OF THE BASIC VARIABLE 
TO LEAVE THE SOLUTION. 
? 6 ' 

NON-BASIC Z(j)-C(J) 
VARIABLE VALUE 

1 -5.5 

6 ^3.5 
5 -3 

4 . ' 2.99999' \. 

IS THE SOLUTION OPTIMAL ? 

? N": 

ENTCR.THE^. NUMBER .OF THE NON-BASIC VARIABLE ■ 
* TOU' "WANT TO BBOTG. 'IN5^ . . ; - - ; '.v:^ 

BASIC ^ ■ PIVOT A ■ - ' R.H.S. 

VARIABLE > -'COLUMN * - ■ * VALUE 

-;3-V-; ; ■ .1.5 ; - .^ -,5 . 

2 "2v5 . 2.99997 

7 ' , 6.5 * • r 19 



R.H.S* 
VALUE 
6 

2.00002 
2A 



THETA 
VALUE 
6 

6.5. 
30 • 



THETA 
VALUE ' ■ 
. 18. 

3.00002 
14.4 



THETA 
VAJ-UE / 

3.33333 
es-l.a9999 
^2.92308 
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EJfflMPlE 3 (cont.) a ' 

•■ ■ ■ ■' ^ - ■ ^ , . ■ ' 

ENTER THE NUMBER OF THE BASIC VARIABLE 
, TO LEAVE THE SOLOTION. 
• • ■ ? 7 . 

.. . • NON-BA§IC , Z(J)-C(J) . 

■ VARIABLE V/^UE ' ^ . 

♦ ♦ ; 7 -846154 
^ . . 1-38461 

^ ' .5 - --461538 ' 

. • • V ■ : . 4 ^ . 2.15384- . • 

IS THE SOLUTION OPTIMAL ? 
;: ■ ■■ ^ ' ? N - ^ " : ■ • 

? . ENTER THE NDMBER OF THE NON-BASIC VARIABLE 

YOU WA^JT TO BRING INTO THE SOLUTION. - ' ' ' 

■■ ■ ■ ? 5 ■ ^ ■ . V .. 

■ ^ , ; -. .BASIC . PIVOT- R.H.S. THETA • 

. , VARIABLE.- . COLUMN VALUE . " VALUE 

3 - .307692 .615385 <i 

^ -.846154 10.3877 -12.1818 

'1 « :461538 2.92308 6^33333 

ENTER THE NUMBER OF THE BASIC VARIABLE . - 

TO LEAVE THE SOLUTION. ■ -J ' 

■ ? 3 ■ 

NON-BASIC Z(J)-C(J) ' ' , fl. 

• . ^ VARIABLE . VALUE ^ 

■ ^ 7 ; -.499999 ... . -r 

6' - ■ .1^5 ■ ■ ; ■ : • 

31 i.5 

A 2.5 • . . 

IS THE SOLUTION OPTIMAL T 

? Y . ^ " 

PINAL SOLUTION VALUES -AT TERMINATION . ; • ■ 

CURRENT . BASIC SOLUTION^ 
- ROW , . VARIABLE VALUE . 

- .1 ■ 5 , . . .2 

^ — 2 • ^12 

3 . , . 1 ■ . - 2. 

■ OBJECTIVE FUNCTION • 54. a 

R; T^0.52/S.95 15t28:J.6 • • i • 



1 ■ • - . . r ■■ ■ ■ \ ; 
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!• : INnODQCnCN 

. In this paper, w& discuss a system for solving 
unconstrained nonlinear least squares pcoblsns. 
The' problqn is dfifinrrl as follows: 



nan F 

X 



where the f ^ are .dponlinoar functions of the param- 
eters Xj /X2/ ... #3^. ^ A special case of this problem^ 
of great practicaJL in^ortanoe/ is the npnlinf^y ^ • 
regression prohlem, where the f . represent the 
residuals obtained by fitting a^nctilinear model to 
es^erimental data. 

The motivaticn beiund the develcpaent of this 
system . is to papovide the user with a broad range 
of facilities \4uch he can activate to ultimately 
enable him to obtain a solution wltitx a minSTi tiT ^ \\ 
expenditure of ocngcter. time and liis own time, Vfe 
s u bsc r ibe to and attenpt to cxtena^Si"5^Ic«pphy 
given by Aird [1] 

■ ■■ The four major goals of this. proJbiem solving 
system are: ' ■ ■ : 

(1) 3t> give tiie user Ji niljuLLUd ticp about the V 
' brfiavibr of hisr fu nc tion in a reggLrcTwhicfr he ■) 

sp ecif 3ct ;:. that is # at a set bf^ points imi- . 
.'fotmly-distribated'^tooughout the region, ^ t^ 
furnish the user vdtii a sanpling of F as well . 
as with certzdn gradient and Hessian infbrr 
mtinn — when this infoaanation is needed and ^ 
used an^wE^ by other bcn^xaients of lie p^ 
lem solving system, Cfae of ^*e disadvantages 
of .many.ex1.sting tsptixniCTtion algarithms is ' 
that when they do not. oonvexge, the user is 
left wilii little, if any/ information about 
the b^jaivior of F in his region of interest. 

(2) Tb give the user assistance fron the system 
•in efapo sin r r good starting points, Many nan-' 

. linear, models are so oot^lex that the 
. scientist has little advance Xnowi^(3ge of €he 
' location' of the n;-> »--!Tmtn 2 - i^fj ^ »^>f -f»r values, 
.Even in'Cases.r4<here.tte sc^^ 
.physical, biological /etc,, obnsideratians) a 



; good estimate of the pptiraal paraneters, say ■ 
to within one order pf magnitude, F might have 
a number of xnaximar minima, and saddle-^ints 
close to the cptiiann of interest, (A saddle- 
point of P is a point where the gradient of F 
is* zero, but F is neither a mnxiTngn nor a 
mimmim,^) The syst^n provides the user with — 
tlie ability to specify an entire olosed region 
i^iich he believes, contains the ^m rr ^j n in- 
steaid bf forcing him to specify a single 

'\ starting point. All tob , often a single . 
"xou^" starting point can pcoiixie diviesTgeiice, 
or even convergence to a local minijtun far 
frcm the minimum of interest.' Cuce tbe bounds 
6f the region have been furnished by. the. user 

^ ii^jut, this system autcm^-'^^Tly ria-rrrw s-n 
on a smaller but highly premising region^ in 
which good , starting points exist, 

(3) .Oto ufiliTTi^ a ni .' gn ber of Qptimzation methods so 
as to solve those problens for which particu- 
Xar Gptimizatian nietbods, fail to find a 

, satisfactory solution or per fa r m poorly. For 
* exaxsple,' whereas a Levenberg-ilarquardt t^pe 
. of: method (vfaich uses Jacobian liatrix infer- 
> nation) might be vrell suited foe a certain 
. scientific ,mDdel throu^rwt moat of a particu- 
- lar region,, a. vastly diiferent type of methnd,, 
~say;a ;Siiiplex method i^iich uses no partial 
- ; • dterivativei'infpoatiOT (only .function values) / ' 
1 mx^it be mdre appropriate .for • sccie parts of ' 
J : 'that 'region, ' * - 

(4) .Tb systematically and autonatically find the 
^lobcil mi.m'Tmm »in the user's; region of- 
interest, (By "global minimum'^ in this p ap er, 
we mean the fullest of the local minima 
values in the laser's region, of interest,) 
•This is aocccplisheti in two we^. First, as 
described ^bove in (2) , startijag guess candi- 
dates are successively narrowed down to ones ' . 
most likely to yield the global minimum, 
•Second, even after successful convergence to. \ 
a minimum frcrn one or mere of the starting 

. . points, the xiser can pre-specify that ad±L- 
tional premising starting points should be 
, pursued in the>atteipt tp^fand the global- . - 
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At present, the algorithm vAiidilurpIemm 
the basic philosophy of this system is at an exper- 
ijnental level and shcxild not be nustaJoen'for the 
final versicn, nor should the KOTJRftN package which 
. realizes this iiiiplaa:ientation be construed as finals 
code. * ' - 

The ^is^er is organized into six sections, ' 
Sections IX and III describe the general algorithm^ . 
secticn IV discusses the curcenti iitplenentation of 1 
the general algorithm/ section V^presents a test " * 
. problem with numerical iesalts obtained froa the 
FORTRAN package for the current '.iiTTplgnentation, 
and section VI describes future research plans. ' 

^ IJ. DISCUSSION OF THE AU3QRITHM *■ 

1S>e structure pf tfie sysrban solver vas modu- 
larized into- four basic tasks or phases: 

CD' Pre-optiroad analysis 

(2) Starting point -generation and, selection 

(3) Problen solution 

(4) Post-optimal analysis 

^ . / m qrder to inplement the oono^t of modulari- 
zation, a control piogxam was' constructed. Ihis 

. program gives^the user the' ability to select one 
or more.pf the ^bove phases with which to attack. . 
his problem. Bach phase can be accessed alone or 

■ as part of a collection viiich includes other phases. 

III. DESCIOTTrCN G^TSE PHASES , 

Phase 1. The fi^ase ■ ^Aich we call "pre^-GptimaO. ' / 
' analysis" ihcor^crotes the ideas of (1) pre- i- ' \. 
scaling of the^prablem, \ahd (2) verification «iid 
initialization of •'user^-su5:plied routines, xii par*- 
ticular, the verification of "userj-fumished exact 
an2Q.ytic partial derivatives by means of finite- 
difference techniques. 

PhsLse' 2. 13ie motivation behind phase two. is derived 
fran the situat i on a user faces >ten attadcing a 
; pffoblan about which he^ has' very little infcnnation* • 
He 5^^^_nat be able to choose a staSrtiijjg^int 
.sufficSentd^y 'close to the.miniinum;!-'^ P^a^ at^best " 
'he can' supply" ujpper and lower, bounds "tii^fc^regian . _ -* 
- in which be suspects .a.ininiitjum to ^BadLst. In order - 

■ to handle situaticos'sucA as thiSr^ a point-disper^ 
sion algorithm can be used to generate a specified 
number of 'points which are uniformly distributed 
in a closed rectangular regim that the user, has 
defined. ;Liet the set s = {sx^Sj,... ,s^ } denote 

these- points., 'Ndw evaluate F on the set S and select 
a subset, T= {ti,t2,..-,t^ } c s, with <^Ni, 

which consists of the ordered points for which F 
has ascending valines; i.e. 



F(t^)^< F{t2> l..^< F(t^) • ^ 

TAt 'this' stage, t^r appears tb.be'our most promising 
■point for producing a minimum of F; .however, as* 
Figure 1 (for "a one-dimenisidnal prdblem) ^xms, ti 
mi^ht not lie in the valley cohtaining the miniimiii 
of interest. ' 




-I h 



^* ■..■y X .t2 t, 

• 'Figure 1^ ' 

'All too ma^y problem solving systems ^would iise tj 
as a statrting point for an qptMizaticsi procedure 
which would then converge to x. ^- Our approach 
^ follows that of Aird [1], ndnriyX from each (jf 
the points tx,t2f we lie our; best ^ * > 

(fastest) optimization algorithm for a ^tiall ntxnber 
of iteratio n s (we have used three for this nmber ' 
' with excellent success) , -producing yet a third set 
of points, P =• {pifP2f*/Pj^ }- Acpin, we evaluate'. 
F ati the set P arid select a .subset 
R= {ri>r2r-*-,3^ } c *P/* with N3 <_ Nj, vdiich con- 

.sists of points ordered so that Fixi) <^ F(r2) 
FCr^^) • The points in R 9an^then be used succes- 
sively as "final starting points". . In practice^ 
■ the rjBstrifrted set R has yielded excellent., starting 
points for use in phase three. -In the figiice above, 
.■ -please note that - any good gradient (descent) 
■cedure.wuld result in rj = x* beirg used as the 
first (most pronising) starting- point in jhase 
.three. V - ■ - 

Phase 3* The methodology of phase three concenr- 
trates on problen solijtLon: convergeiice to a 
minimarii nvro major .considerations hsve arisen in . . 
^the developnent of ' a vgenera^ . ' . > 

(1) • What strategy can be used to bleid ' 

^ eocisting cpipleneiitary optimization ^ ' 
: ' ' ^ : ' techniques in sudh' a <tecy as^to achieve ^ " 
. a balance between^ efficiency, and- robust- 
ness? (By "efficiency" we mean miniiniza- 
tion of speed of execution and core stor^ 
' age requirements- iBy "robustness" we 
V niBaa the ability of a mathpd to ocsnverge - 
fran a very wide distribution of start- 
ing ^ints for a 3;aiiety of functionjs.) 

(2) 'What criteria are involved in disoon-, 

timing the use of one method and 
■ initiating. the use of another? 

Ideally, we would like to use one method that 
is;both. efficient aixJ robust:.' ll&wisver, highly 
■/efficient loethods are seLdon robust, and vice , 
• versa^ A lornl optimizer is r^^^^^^ie^ as an . 
. efficiisnt method and is one which converges very - 
rz^ddly for starting points that are dose to 
. (lo^d to) the minimari. An.exanple of a local 
•^optimizer is tte G&uss-Newtcn method applied to 
finding the miniranm of quadratic ftancticn (fran" 
aoj^ starting point) . . In co ntra st, a robust method 
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win usually acuv ^ SLye to a liiiLution froa a wide 
oollec tiqi of starting pointSr alttou^ the cx»- 
vergeioB rate is generally nuch slower than 
using p. local optimizer. An eson^ of a roJbust 
. method, i**. ^ nonlinear siaplcx algoritha. 

These Oonadderations led to the develOEnent 
of a hierarchical stnjoture in \^hich the most ^fi*<- 
/ cient method is at the top of the hierstrdiy while 
less efficient but noce robust methods 'follov, and 
finally/;the most robust method is at the botton of 
the hierarchy. JIhe structure we have adopted 
utilizes an efficient method'as the primary method 
' of problan solution. However, if the efficient - 
method falls tomato reasqnable progre s s towards^ 
the minisuo/ alternative methods which are -less 
efficient but more robust and which have caopleteiy 
different structural properties should be available. 
Ftqr .exanple/ it malces little sense to switch bade 
and forth betMaen local methods such as Gauss- 
Newton, Leveri b er ij Ma rcyaardt , and Steepest Descent 
if the Levg3berg--M ar q u ardt algorithm is not con* 
verging properly, sinoe the. Xevenberg^torquardt 
algdrithm .is already a ccvpEanise betMsen the other 
tMD. . Ir^tead, cnQ should ^try a method having a 
ceD^alete^y different -Structure, ^pe3;h^, in this 
caser a dtf^-:Newton method or one of the better 
heuristic search procedures. 

-in our inplanentation the local mgthjd is 
used and reused first, and only wben .tte local 
met hod fails to. make progress do we svtdtc^ t6 a 
m o r e rcbdst method. 

Phase 4. In phase four, whicii we "post* 
cpt MTB l analysis*, it is verified' «yhether or not 
the solution obtained in phase three is indeed 
the Tnlnmm,' as opposed to, say, a saddle-point* 

IV. ^ CORRaZP IMPIOEOTVnCN 

The current implementation of the system 
. solver is described below with the aid of the 
* flow diagrams shown in Figures 2 through 5, -* 

ate ocntrd procpmm (see Figure 2) directs 
. the of ocntrol of the sysban^ ttis is 
. ji n mtT>T i^3ed ty^ two input parameters, PPPT and i 
. OCrroPT, eadi of tates cn a value fcon cne 

to four. The valoB. of INCPT ^jedfies which 
phase the user wants to access first, vhile the 
^'-^ value of.ODIOPT. defines the last phase he wants 
to aooess. Pbr example, if ^^act }jsex ree^jested 
exacuticn of all four phases, INQPT would be set 
to one, and OOIOPr would be set to four. However, 
ifhe only wnted tp generate starting points^ both 
HCPT.and OOTCPT would be set to the value of two/ ' 
. The ooptrol pcogran also enables 'the user- to 
: ycicif y more "tiian one starting poin£ when attenpfe- 
ing to find the mininun. (These points will be 
.used en s nnr^ emiv e aL LeniJ ts by phase tteee.) The 
ability to use more than one starting point has a 
two fol d advantage: 

(1) It increases the possibility of converge 
\iDg to a global ndTn'TTun fztm at least 

one starting point* , 

(2) If the user's proUm is very oaaploc 
and it as difficult to find evrn^ 



begin 



/ input 7 

/ iNOCT,ooropr/ 



INOPT 




phase 4 



o2^ 
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' local minlnun, tha poulbility of con- 
verging to a local minimum IncgBaaeB "when 
using mora thin cm atarting point* 

At pcaant, phaaa cm W not teen inplcntented. 



PfftSB 2 



begin 



Gamr a t e K. points 
'and calculate their 
F values 



Sort points on 
their F valms 



EI 



Choose 



2Si5t2^ 



Psrfonn a liadtad 
nuober of .Itesa^ 
tions of local 
sDBthod ficci esiA 
point 



Sort, points on F 
values and discard 
tifise too close 
together ^ 



Save N3 < Nl of the 
mainZng points as 
starting points 



rettrm 
Figure 3 

Ote OQnstructicn of phase two (see Figure 3) 
is scDMhat more detailed ttan its description 
given aborc in section HI, but it the 

aLj o riti in^ K^^ N2/ and N3 are user^ > 
iWOT^ ^ 'O'^ parameters. ^!he. currently used 

in phase tio are the Aird and Rice poinfr-diqser- 
sion algoritta [2], and Brown's derivative-^ee 
modification of the l£tvefi«g^-MBzquardt m [3]* 

?S)e current inplementatin of phase three 
(see Figure 4) utilizes a local optimizer as tte 
prijiary method far >y>lution and an alternative 
method, a nonlinear sinplex algoritlm. Olhe local 
method is used. initially for a specified nunter of 
iterations. lAan 'the jxethod does not oGRvesge to 
a mininun, tiie progress of method is evaluated* 
If the results indicate that it is pMrrn t mi Tyj ^^ell 
enough to. continue, the floiif of ocntrol returns to 
tbit 1cra7 optimizer. If the ptm^rt^m <*yw i^atiq n 
indicates that ^the method is perfbirming poorly, 
the nonlinear si2splex^ method is eaployed. if this 
method does not oooverge to a Tnin-iiniTrt iji a pacified 
mnber of itieraticns, an evaluation of its 



FmSE 3 ' begin 



enter 
local method 



good 




return 



.enter 
sjjiplex method 



good 




return 



^ return 



Figure 4 

^ ■ 1* ' ' 

performance is necessary. Ote flow of control re- 
tu^ to the local , optimizer if the resulti^indi- 
cate a sig n i f i ca nt degree of progress. However, if 
.the sijcplex method has perfSozmed i- mg^t^s g^f ^ irTl ly/ 
- the flow of.. oQtitrQl. returns to the oontxol program 
with an. inrTi'mticn tba^t a solution oouXd not be 
foond. At this point, }^ next most paxxuf&dng 
starting point candidate is use$^ and ]Chase three 
is re-entered. Suocessful* progi^ass of ^both opti- 
mizers is depiendcnt jjpcBci a significant decrease in 
the F value or in- the norm of the gradient- Ibe 
local inethod""3irrently used is Brown's method {3]. 

nonlinear simplex algorithm used is P^jdcinacn's 
itDdified version of Nelder and Itead^s* nonlinear 
siirplex algorithn £4,5]. ^ 

Post-optimal analysis^ in pteise f our (see 
Figure 5) is aoomplinhffri by excDni,ning points in a 
Z3ei^:borhood of the solution obtaii^ in phase 
three and then oai|ad ring tte F values of these 
points against the F value at the solution point. 



V. A OSSX PKHEIM Am ICMERICAL HESULES 



^ fDRrraN. pjjoyrdm based qpcn the current im- 
♦ pl^mentatim of the problem solving systan has 
' beien run successfully on a oollecticn of standi 
test problans. Clhose results will be dccunented 
elscs*ere.) Bar the purposes of this discussion, 
we have constructed'a new test problem \^»se 
geometry would diallenge the ability of the system 
to find a g lobal minimon* 
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return 
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Figure 5 

Hie test problem is given by: 
\ itoniinize F(x,y) -wtere 

F = .0001 * (f + f^^) / 
. f^^ 200. -T 175- * [exp{-(x - 17-)^) 
'+ expeCy - 17.)^)]. . 
f2-= 5; * [(X - 12.)^ * (X - 23.) 
* (x - 17.) (y -.120^ 



ty 23.) * (y - 170]. 
Note that t is symnetric in x and y. 



, A three-diinensicsial plot of F over the range 
of interest is given in Figure 8. -Hi order to 
better e:diihit the top ogr a phy of F around the glo- 
bal miniina, a o o nto ur $>lot is shcwi in Figure 9. 

T&e reason that F is challenging, especially 
' local cpciitiization methods, is that the regim, 
X = (11,24)' and y = (11,24) , cccitains a number of 
local minijrH and saddle-points, including a saddle- 
point close to the global minima. Si pecif i dea lly, 
the points given (approxiitately) by (12,12) , 
(12,23), (23,12), and (23,23) are an local miniira 
of F ani at those points the valxae. of F is 4- 
Similarly, ^±ie points given (a^rgroxiirately) by 
(12,17) (17,12), (17,23), and -{23,17) are also 
local mln'*™ of F; at. each of these points F takes - 
on the value of .0625 • Tlie troublesone saddle- 
point occurs at (17,17) vtere F assu nes the value 
of 2.25; this saddle-point is near the, global 
nanina.vdiich are given '(appcoxiinately) by - 



(J,7.61014224\l6.110990468) and. the synxretrically 
placed point, (16.110990468,17.6101^224). At the 
globed, minima F has the value of zero. 

An observation to be noted is that even finer 
cor^sour plots than the^^pne given in Figure 9 ^ 
faxLed to expose the iLocatians of the global minima 
Qcaphicetl technique^'^haye merit in shewing the 
rverall general behavioir of a function, but, as 
this test function indicates, graphics cannot be .. 
relied vpan to solve optimizaticn problems. 

agperiinent #1 . In order to obtain a measure 
of the robustness of th^ system salver vs. the 
robustness of - the stand-alone methods o£ vihich it 
is ooiposed, we ran (a) the systan solver, (b) the 
local optimizer (Brown's method [3]), and (c) the 
siitplfec method [4,5] from first 10, then 20, and 
finally 40 starting points unifonnly distribcCbod 
in the region of interest, x == (1,31) and 
y = (1,31) . the results are sunmarized in the 
table given in Figure 6, . _^ 
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^ Figure -6 . • 

ate weed "suocess" in that table means that tiie 
.method converged to a local minimum (which in sane 
cases corresponded to a global minimum) , v«hereas 
the ward "failure" means that the method aonveirged 
to a saddle-point, d iv m^ed , or failed to converge 
in the naximan mmtoa: o£ Sanction evaluations or 
maximum number of iterations that were allcwed. 

■ Eacperiment #2 . The purpose o£ this e^^ieriment 
was- to see how many points (Nx ) had to- be uniformly 
scattered throu^iout the region in order for the 
■ system solver to produce a globeil mihiTnan as dis-* 
tinct frcm a local miniiruin. As Pigure 7 indicates. 
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as socn as 20 points (or nnre> arS initially 
scattered, the systan splver converges to. a global 
minima. In carder to illijustr^e this es^seriment in 
greater detail^ let,us^ consider the case in which 
Nj = 20'> N2 5, and N3 = 1 in Figure 7 (see also 
the discussiari of phase two in Section In" of this 
paper) . In phase tsJo of this - algorithm, an 'ii^t 
of Ni == 20^ causes 20 points to be unifocniLy" ' 
. scattered iy the Aixd-Ribe algorithm [2] in the 
region of. interest vfcich is x = (1,31) and 
y = (1, 31). "The location of these points is 
shewn in Figure 10. Olie nurchering. of tte:points \ 
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Figure ,10 

given 'in Figure 10 corresponds to the order in 
•which they were produced by the Aird-Rice Method 
' [2] . -'•Phase two, now evaluates F at each of these 20 

points, tche inpit. parameter value, N2 = 5 (see 
■ Figure 9) new causes phase^tso to tate the 5 points 

(of the original 20) having the .snallest F value 
- and perform 3 iterations of the local method fron 

each of these 5 points, Qice sbgain, the T values " . 
* of the S.resulting points axe. cccputed, ^Finally/* 
the ii^xrt, N3 = l, (see. Figure 7) causes the selec- 
tion of the one point viiich has thus far produced 
the CTiallest F value to be used as THE starting 
■point for phase three. That "point caused ptoase^ 
three to converge to the global minimum -^^^ ■ 
the yalxie of zero. It is of interest that the* 
point (19,10) labeled" "6- in Figure 10 had the 
anallest F value of the original - 20 scattered, 
points, but if that point had heen used directly J 
as the starting pomt for phase three, the system 
solver would have'^^xnverged to' the local minimum of 
F at (12^12) wilJi the correspoiding F value of 4. 
On the other hand, th% point (19,15) labeled *^3" in 
Figure 10 only ranked third on the list of the 5 
; best points, .based upoci F values only; however, 
when-that point benefited frcm. being run, through ^ 
the 3 iterations of the local method, the resulting 
point had. the smallest T- value^'and was used in 
phase three, whic^ then produced the global minimum. 



(17.61014224,16.110990468) .vdth a corresponding F 
value of 263^. 

■ ■ * ■ ^ " .. 

suranaryi the system solver- di^layed a high 
,rate (see Figure -6) and, once enou^ points 
werg^-t^ed" in the initial scattering by phase tsao 
^ (i.eV'i^-once Nj .was large enough) , 'the system solver 
found ■ a. gio^^ minimum (see Figiare ^7) . 

yi. FUTURE RESEAHCE AND PMNS FOR TEE FIRST 
• DISroiBDTED OGOE " • . " ♦ 

The long range plans for the problem solving 
system include: 

(1) The ability to solve nonlinear uncon- 
*. stLriinerl optimization problems and non- 

linear systems of equation problems in * 
addition to the current ability of solv- 
ing nonlinear least squares problens. 

' (2) The ability to inpiement any of a nanber 
of hierarcSdjcal structures in ptese 
. ' three: Initially, we shall explore a 
■ : three-tiered structure .in which ^the top 
level (the level which is xased and re- 
. used first) includes methods which re- 
. quire Hessian infbmation. The second 
level of methods utilize gradient infor^ 
roation and the bottom level nethods 
utilize only function values. TypicsiUy,^ 
the mettods at the top level are the ^ 
i most * efficient ^^lereas the^methDds at the 

bottom level are the roost robust. , 

. (3) ■ The^ ability to plug in any local method 
or xctoust method into tte s^jpropariate 
boxes in phase three without the user 
having to repcogram any of the rest of 
the problem solving system. Similar 
abilities apply tQ phaises two and four. 

(4) Tlie develQipnent of a control language to- 
. allow the user to make requests of 

system in sziaple meaningful 
staitements* ' ^ 

(5) The creation of a fully modularized 
piackage. 

In order 'to achieve, one goal of producing and 
distributing useful code iy the end of- 1977, we 
shall concentrate on ai^speclfic prbblan area, the 
unconstrained miniTni7;^ticn of a sum of squares of 
nonlineaia functions. We have adopted the follow^ 
ing iinplementaticn strategy: 

Phase 1 . *^ we plan to allow derivative or 
derivative-free methods in phase three, we g>-^ii 
irrplfeinent code to, at the user's option, verify the 
correctness of the partial derivatives vrtiich he 
furnishes. Agciin/ at the .user's qptim, the sys- 
tem will autanatically provide' default valines for 
the required ir^ut parameters. Finally,' a scaling 
strategy involving the diagonal . of the inverse 
Hessian matrix of F will be. tested, aixi iir^JlenEnted 
if successful. . • 

Phase 2 . This phase will renain as it* is 
new, xa t Jl iz ing the Aird-Rice point-dispersion 
algorithm [2] follo<*;ed by .three iterations of ■ ' 
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.;9axwn's:inethoa [3] , . . " 

HvHse 3. Wb,. shall adopt the tfaree-tiered 
.hierarchy/ using JBrcxm's Igvei±ier g^ M arc^iardt • typa 
"of method; [3] at the top of the hierarchy/ the . 
'method of descent . (searching in the direction, of 
the negative- of the' gradient) ikt^the second level 
and again "Parkinson's nonlinear sinplex inethod 
[4,5] at "the bottoa level. , '^^Ihe flew of control 
•which we, shall iiipleroent in phase three is ginfii 
in Figure 11- ' . 
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Note, The reason that Brawn's method [3] and 
the method of descent transfer control to the 
sinplex method upon successful convergence is to 
avoid the difficulties associated vdth saddle- 
points. Cur- numerical experiments h^ve indicated 
that gradient and Levenberg-Marguardt type of 
methods will readily converge to a saddler-point; 
hcwever, a few iterations of the sirtplex method 
are sufficient to move away from such a point. 
C&viously, there wi3J. sufficient tests made to 
avoid infinite looping (lapon successful convergence) 
in phase' three. . 

Phase 4 . We shall 'canpare an Airrd-Rice [2] 
scattering approach with a cyclic coordinate search 
in \^4iich the coordinate axes have been rotated to 
correspond to information obtained from the statis- 
tics of the nonlinear regression fit. rSiB best of 
these approac h es will be used in phase four. . " ' • 
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Abstract 

. . The purpose of this paper - is -to present a ■ 
description of a system of . subroutines to, compute 
solutions of the .'it eratively ■ reveight ed least- • 
: squares • prohlen vhere' the weights . themselves : are ■ 
. functiions of the scaled- residuals .'^ . Starting 
points* for. 'the" iterations /are -the- ordinary least 
. sqxiares • solution, the overdeterniined solution ■ in ' 
the.-L- norm, or starting points- specified by the 
user. ■ ' ■■' 



■ ' lnt,r oduction ■ . - ■ 

' ' ' '■ 

The' iteratively reveighted least squares. 

■ algorithms • are a part of robust regression vhere 
' "robustness"' is used, in -the stat^istical -sense of 
. .relative -insensitivity to moderate* departures^ 

. from' assxrirpt ions ^ ■ The .exp^iniental . systest.'of 
• subroutines .i;hat ;is" used to j^ccanpute the solution 

to the iteratively reveigh^S^li least squares ■ * ^ 
. .probieii' is modular inatbemat^t^l •softvare--vritten- ' 
- ;as-.a collection- of- Fortran" saifbroutines ; :. The 
■•■'subroutines are designed tq, 'operate efficiently. - * 

axid -r^liahly on .computing -laachines ' of the' major ' ; 
' : manufacturers . The ■ specific machines' .to which 

we refer ^e ■CDC'-6600/T.60O, Honeywell 6000^ : 

IBM 360/370, PDF -ig^^ and Uhivac 1108. ■-. . -, 

: \ * The ■ software for solving' iteratively 

■ -reweiehted least" squares problems represents 
interdisciplinary .research in" numerical analysis,' 
robust -statistics and qxaality software and, as 

_ such, represents the. combined; work ol, many .' 
people. • The "basic, design of .the iterati'vely* ' 
re^^eighted least squares algorithm and the 

"computation of the* default tuning constants''^ 
for, the various Veight 'functions was done "by 
Paul "Holland. The convergence criterion", for' 

.iteratively reweighted. least squares was 
devised by John. Dennis* The start from 
the overdetermined solution in the L. ■norm. vas. - - - . 
provided^ by 'Richard Bart els., .The. subroutines ' . * 
for the software for the stem and leaf display , 
were done by. David Eoaglin and ^tan Wasserman. . 

•The-, design .of the interactive- driver program was - ■ 

"hased'oh the advice of David Hoaglin and Roy Velsch. 

'•Technical and programming assitance was provided" 

'by David Coleman, TTeil Kaden, and Sandra Mori arty- 
Valuable discussions continue. -tO' be held yith 

■fThis . work was supported in part by the National 
Science Foundation^under. Grant' No ,•■ MCS7i5-11989. 



■ David Gay and Richard Hill. • 

.\; Ihe- subs-fcantiai-: 'contributions of .'Gene Go3lib- have 
. , been, central to this work. His encpiiragement --co us", 
his constructive work on ,the numerical stabili.ty - -■ ■ 

■ of the" algori-tbms,- and his continuing exposition " 

" that" crosses the boundary, of robust stat-istics and 
numerical algebra constitute- an essential resource^ 

Section 1 -. • \ 

Th^ method of least squares -.is versatile and • 
numerically stable when computationally stable 
methods, are used>'. i.e. ri,i*]-.' "Nonetheless, least "vi^ 
squares does not give very much information .about . 
outliers or leverage paints if one looks >s imply " at ■; 

. the craefficients ..x of b -= Ax + e. ' JEn'.the. v '*" 
notation -1^ Ax + e^ h.; is - ah-^ mxl vector of 
observations;. A' -is an . m^i*^ -.data or • design' matrix,- 

' X is - .an .'nxl'- ., vector . of parameters ' e . is an .- mxl' ' 
vector.. ■ We ;recognii2.e".that the usual .statistical .-. : *:. ■ / 
■notat;ibn.-is..y. =: XB.+; ^ '/where y. is • ■ nxl; X" : is . . ; - 

* 'nxp.v -,B . is -*5)xl,. -"'and 'J'e, ' is nxl. ; In the;; "-:■■ ■ 

; Iteratiyely reweighted ' least sq.uares * subroutines" 
the ; software, for least 'squares model fitting : .' 

-. te?hnoiogy .has been* extended to provide' more 
information about the data and to provide a vehicle'- 

■ for. handling -large residual problems 

, The -/ordinary- least squares problem /is 




' where 

r is tfee residuals, b-Ax, and s is a scale. 

The. weighted least squares problem 'is 

V , , ' . m' - f r. (x) 

' min I ■ W": \|' ^ ' 
X i=l \V ^ 

■ . . „ ' • ■ 

Vhich is solved by using ordinary, least squares 

with •W . is a diagonal matrix 

of weights thaVaa=e^^^\inctions of . s.caled residuals^. 

The iteratively reweighted least squares 
problem assumes a star,t- Presently the L2 start 
can be' computed from'MINFIT from EI€PACK II 
followed by MINSOL which 'determines \he b6st. • 
approximate rank of A and computed :the »I^ast ' ■ . . 



squares '-solution. Alternatively the- start can 
"be ■ cc3inp\fted from the suWoutines. QHF, an brtiLOgcjnal 
decomposition ■based on Householder^ tranf ormations , ■ 
foUowed.'by'QPSOL vhich solves Ax' = h from the 
output of QBF. An Lt^"" start can .bie obtained from 
the^-SUhroutines^Lj., by Bichai:d 'Bartels , vhich 
compute the • overdetermined' solution in the L^^ norm; 
Given the starting solution, the scale is : deter- 
mined from- subroutine SMAD to 'get ti^e- median 



r. ¥. 0. The scaled residuals are 



forSned by subroutine SCIMAD. , The weighting matrix 

'is determined from any one^of- the eight .sub- 
routines that coffiT^ute the -we-ight functions . Thus , 

given X^^ .from Lg. or L^, the, problem is iterated 

to obtain' ^xf^"*""^^ =' (A^W^^^A) V^^b using MBJFIT 
or QB factorizations- ' ' 



To test -.convergence, ■■ after the k 
ve compu'te- ■ :■ . , 



th' 



iteration,; 



1/2 



.where j ] •.|.|.. " . is the Euclidean norm. 
'Subroutine' WGRADl computes the gradient and srrb 

L^e WGRAD2 .computes -a; scale .independent measure 
*'of th^- gradient'. •* ' *. ' 

Section^ - . ' . - * * . . ' . . . . 




^ - . The .'feerm "robustness'V has 'a' common ;thread d 
-y eanipg .^t^iai* carries . ' through ' " st at ist i cal* ' roisus t ness , .. 
omputational stability, and reliable -sof tvare* ■ 
From th^' standpoint .of reliable software, 'moderate 
.epartUr'es from assumpt?ions .means 'that the per- " 
•rmance" of the spftware shall' 'be unaffected (in. ' 
th^s ens e that per f ormanc e . wi' ll not be ■ dejgraded' ) 
by. the "enyironnent" in which* the - softvare is run, 
.the compiler .from which code is gener;ated, or ' 
the^ppli cat ions system in which the software is 
ijnbedded- ,Iii particular, we program t'o -avoid ■ 
abnormal system intgftmption or termination. 
Cody iz\ has .giveqj^'^ excellent exposition of 
reliable software and has provided more details 
'in his position paper for the workshop on robust . 
sofl^ware,. Computer Science and Statistics, 
Ninth Annual Symposium on* the Interface. 



. Throughout the ;work on iterati'vely reweighted 
least squares heavy . emphasis has been placed on 
modular subroutines. . For example the convergence 
criterion can be changed," weighf' functions; 'can be 
added, and numerical ecjuilibration (for- columns ■ 
of the .A' matrix) can- be' invoked. ". OptiohalJLy the ■ 
."hat" matrix -which is the .projection' matrix, 
- « - ■ ■ ■ . 

" .P. = A(A^)^ A^, . • 

- . • ^ . • 

is obtained, as -from -the' singular yaJ^ue 

decomposition .or ■ QQ. -from Househplder tr-ans- - 
formations. ■ If ■desired, the stem-and-leaf display 
of the residuals. is -provided. Ah- interactive 



driver program' is- used to print the singular values j 
the..L2 condition number, select one or more weight- 
ing functions , display residuals , monitor, conver- • 
gence, and optionally select the' diisplay of .th^" 
. diagonal or th6 upper triangle of the^ "hat" matrix 
_.and the histo^am for the stem-and-leaf - 

. . ^Our- ^.pproach to programming design included- . • 

■ -documentation . for .use, and flov' of program control, 

as comments in ^the program. We use a declaration . 
checker to be "sure that all variables have been 
declared. The Fortran verifier, PFORT-,- .from . Bell 
Telephone LaboratorieSr, was used to check .the sub- 
routines, and the Fortrem converter , - from IMSL, ' 
was used to,, generate the Fortran code for non-rlBM •• 
machines . - ■ " " _ 

The explicit weight functrions that ve have 
'listed are. listed in the Appendix I, Table 1, The . 
-subroutines,^- with the exception of those' reqiiired "" 
for the start, are listed in Table 2. Typical 
convergence quantities, for da.ta ..from [3] are . 

■ listed in Table 3-- Appendix II' shows a sample ' 

■ experimental .program for', pne. of i:he' weight- functiors 
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Examples of weight functions (where u = scaled' -*' 
residual)' and the^default tuning constant for. r 
each weight function. • 



ATORSWS 



sin ^) . |u < TTA 



U > TTA 



A.= 1.339 



3IWSIGHT -wl(u) .=^|l 



hi. <:b • 



• |u >-3" 



c « 2.385 



' FAIR . Vp(u) = 



-F = l.ltOO 



u < H 



HGBER ' .; 



^(u) =■ 



1^. " |ul ->.H 



LOGISTIC- V, 



tL.= 1,205 



hi < ^ 

\m\: > T . 



T = 2-795 



WELSCH- '* -v-(u) = e 



R= 2.9&5. / 



t 
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NAME. 



DESCRIPTION 



EQOl - ' MODIFIED ROW-INP-EQUILIBRATION v 

EQ02 • COLUMN {MAX. ELEMENT) EQUILIBRATION J,. 

EQ03 ROW (MAX i ELEMENT) EQUILIBRATION 

EQ04 COLUMN {SQRT. StJM OF SQUARES) EQUILIBRATION 

EQ05 ROW (SQRT- SUM OF SQUARES). EQUILIBRATION 

EUNORM EUCLIDIAN (SQRT. SUM OF SQUARES) /NORM . 

HHAT FORMS DIAGONAL OF H-MATR^ ' {U*U-TRANS ) • . 

HMATQR FORMS DIAGONAL OF H-MATRIX (Q*Q-TRANS) 

ISORTl ^ SHELL SORT (DECREASING) USING INDIRECTION 

IS0RT2 SHELL SORT (INCREASING) USING INDIRECTION 

MINFIT . SINGULAR^ VALUE DECOMPOSITION A=U*SIGMA*V-TRANSP 

MINSOL SOLVES AX«B GIVEN OUTPUT FROM MINFIT 

QRF - " QR DECOMPOSITION r .Q ORTHOGONAL TRANSFORMATIONS 

QRSOL SOLVES AX=B USING^^gRF V 

RESIDL '■ 'COMPUTES REDISUAL B-AX 

SCLMAD SCALE RESIDUALS BY SCALK3G FACTOR 

SU)Spy ' DOES STEM AND I£AF DISPLAY (CALLS OTHERS) : 

SILEAF DETERMINES STEMS AND LEAVES 

SLPRNT * •^^PRINTS STEM AND LEAF DISPLAY 

SLSCAL" . DETERMINES SCAI^ FACTOR AND UNIT FOR DISPLAY 

SLSCRT SHELL SORT IN INCREASING ORDER - . 

SMAD DETERMINES MAD SCALING"^ FACTOR 

WANDRW ANDREWS WEIGHTING FUNCTION 

WBIWGT.' BIWEIGHT (BISQUARE) WEIGHTING FUNCTION 

WCAUCH - CAUCHY WEIGHTING .FUNCTION; \ _ . 

WELSCH * • WELSCH WEIGHTING FUNCTION 

WFAIR - FAIR >?EIGHTING FUNCTION . 

WGRADl ' COMPUTES GRADIENT 

WGRAD2. COMPUTES SCALE INDEPENDANT MEASURE^pP GRADIENT 

WHUBER HUBER WEIGHTING FUNCTION . 

WLOGIS LOGISTIC WEIGHTING FUNCTION 

WTALWR TALWAR (ZERO-ONE) WEIGHTING FUNCTION _ . 
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. The* data ftrom .[3]', suggested "by Paia Holland 
..as. test, data is a 25: x 10 data matrix A vith: 



A^^ ='L.o: 



The- L^" condition mimber 'is 



' ^max _ .396 X 10^ 



-1 



The oaxiinuzn diagonal 



• min" .789;x 10' 

element -.of. the H, "hat" matrix is .85- Based 
on functfona of the. scaled residuals, jthe effect 
of the weight ■^^lnctions^^^^^^ some 5: 

the- observations • 

• For the weight functions listed belofw the 
^^ m axTn r um ielement* in magnitude of the scale-free 
measxare.of the gradient after iteration 1 and after 
iteration 10 is. as. follows'. 




after iter 1 after ' iter 10^ 



Andrews 


^1 


Start 


.383 


'.2kk X 10'^. 






.start 


,.iov 


• 177 X 10"^ 


Biweight 


^1 


start 


. 383 


.2l;5 X lO"*^' 




^2 


start 


.105 


.181^ X 10"^ 










'.609ix ibr^ ' 


.fiuber 


^1 


start 


.383 ■/ 




^2 


^art. 


.89^ X 10"^ 


A35 X 10"^ 



/ 
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TMTPGFR T 
«««4r«FnNCTinMS: . 



> • • • 



t xs : 
»- * 4 ft 4 



THIS snKftourTNF PRnrkMCPS THF 50II&RF «»onTs OF tmp wPtwrs 

hPT«=RMTNPn HV THF fWPIlT VFCTOP 11 OF 0<FV|aiinLY . r.nMPMTFn . ; * 
SCALED RFSinilAL^ AND THF R1WF)GHT (HISOIIARFI WFItXHT' Fl wC. riOv. I 1 ) 

- . ■w ■ ■ ■ , - , -. ■ ' _ \ ■' " ■ . ' ' 

OM. TWPMTr _ 

M- Mll?;.T. BP 'i^PT Tn THF ' WII*iHFI« OP F| P-lPwTS TN "fHP : VFc'rOPS ' I hKn\ 

'■5;owt ■ ; ■ ■ ' : ' 

.11 CHMTAIMS THF StAMOARn l^f-f RFSTIMIAI.9 i^RHn A PRPVlltllS I.CMPflR 

FTT. THAT JS» 11(1) « ftCIl /.S WMPRF; f ) f S T^^F T-r f H - 
RFSiniiAl F»n«^ A« tfl^F/fr FTT» «(|| « Yf ll - YP ff TFO C f J • 
Awd,m; IS A RF5}DflAL SCitrMG FlWCTinik* rp.c ^ Cnhf.rS 

RF r-HP (jllTPdX TP tHF FORTa AN SnMRnilTlMF .SrAO), 

COMST TS* THF •rtiWTWG cnKf5*»/wT« FDR Thp tfp|nHT PnNC't ThN- 

WCiH. CRPP AP*>I ICAT>t<f» AWri jKAaP RFSfi^TCrinN??.! _ 



T3N nuTPliT: 



SOW rnNTATMS > VFrTn« OF 'ihP 5011A«F ROOTS OK; TWP wFTGwr*; 

nPTFR^IMPn ftY THF sc«i Fn rp^sT'IMai^s amo thi^ hiPiGwrT^i'^ 
*; ' FiiNCTTriN. ^ ^ ' ; • 

«*WAPPLIC ATTf)r« AND IISAaP R f-5 1 R TCtT 1 flNS : 

THF^^OOT-weiGHTS ARF MPPDFO Ff 'K JHF OImIPII fA rfOKk JIIP fHP 
.TTFRATlVFtY PFWPTGHTFn lT=A5'* ^OlJARFS FSTJMA TFS^ HSTWe; THP . 
PORTRAM SMBRoriTINFfi MTWFTT AKr MINSOL. I >vJ fHTS- CONpii fA f TOM 
SOW{ I ) HMLTTOI TFS THF rnRRfi,<>f'NnTN(; ROWS OF THP )(^mATR TX 
ANO.THFOf-VPCTnR. Ci) 

• : . ■ ■ . ■ ■■">.'.,■ ■ ■• - ' * , , ■ ; ■ - ■ . 

THF LARGPR. THF VALmF nP .chwSI V T.HF MO^P J^CftRLY «LC ^"'^ VAljI^^S 
OF W(t|) wp^l. FOll&t MNTny* ' ^ 

I F CONST tS TAKFW Tn HP VE^Y 5 W AM I f T s «>nsS T h|.F Tn PROnil'^c a 



. MjWtxwiir' 

K f W liKi^fV 
HTW:>fH;-^(i 
. H J w^tori^*) 

Hi Wr»fW»5?V 
HI W 3r>o<6*<' 

rt I w^nnpf) 
-'HTw<^f«''»5<.» 
HI W: 101. on. 
PlWiipJlf' 

:mw;inT.3r 

Kl w<irvlS(' 
''hi wniH6'^ 
ftiw«»m7i', 
HfW-rifii rV> 
W'^rnqo ^ 
Krw>»n7nA 

RlW'VO>l/« 
HtWOO?^'(» 

Hiwoo^^r^ 
Kiwnn7^< 

KIwnn>7o 
kTwoo7ftn 
HTwnn?«>{t 
Hiwno^on 
HtwunMf- 

Hi W0O*^^r, 

Hlwoq^AO 
aiWi>h.^^(i 
ttTwon^6f> 
Hiwnn^?^ • 
HTWoo%R/i 

RTWOO^QO 
RfWOOAOO 

RIwoOiLi r. 
fifwooc^o 

Jki woa^A^o 

Rf Wp04i>O 

t5|wo4^^n 
Blwoo^7<' 
8Jw^in<kfto. 
aiwonAQo 

8jWi>04no 
HlWfrO«>\ci 
B I w^^os 2n„ 

RfWUfv^^lv 
HIW/Vo^AO 

RTWon<>5c 
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VPCTQR OF ROOT-WFIGHTj; A 1. 1 OF UHTGM FOIIAL. 09 WPAQl.Y, FOllAL 

ZFBn,'ANr) TwTS wTLL-BF' USF1.F55 AS; Tfviptir rn thp wF'Tf;HTFO Tfan r - 

<;0IIARFS COMPUTATTOM^. / * . , . 

T*=.A:TnWTNG CONSTANT W^IUF OF 4w<^ftS T<>n«;Fn. 11^ 

ASSMMPT IOW (IF GAIISSTAN FRPPRS, TMF R FSUr T INC FS TJMA THR - 

WTI,C HAVE PFRCFMt A 5YMp 1 J C/ PFF 1 C fir WCY • 

***4:*ALGnQirwM NOTFS: / 

THF INPUT PARAMFTFI?. CONST. CWFCKFn TO AVHTn riNHFRFLOWS AN^P 

PVFP FLOWS. ^ 

*****RFFFR E^C FS:, 
(I) KFA rnN^A.F, 



ANP TIIKFV.-.^UW. ( XQ74) . TFCWNnMpTS? ICS 1 6. 



ji»*wF 197 f> 



4( 4cM I S TO R Y : 
ROSPPACK REI PASF n.^ 
: IF ( IBM) THFW r 

IRM ,3<^n/370 VFRSTON 
FVSF IF OCFROX) THFW 
XFRnx VFRSinw - 
FLSP IF (liNTVAC) THFM 
IINTVAC VFRSION. ^ 
FLSF IF (HIS ) THFN 
HnNf:YWFLL VFUSTON 
. FLSF- IF, (OeC) THFN ^ 
POP 10 VFRS TON 
FLSF IF. : ( CDC ) THFN 
CnwTROL OAT A VFRSTON 
ELSF TF (RGH) i.THFN 
y^UR.«Our;HS VFRSION, 

FLSF, 1;CAR0 

^**i*<c3Ct;^:»<c<c MACHTNF "WFPSinN 
IF -f SlMGL*=^) r CARD.' T CARO' 
STM^LF . PRFCTSTON DFCK . ^ 
OOIWLF PRFCTSTOM OFCK 

' ■ . . * ' ' 

WRITTFN RY NFIL KADFN (NRFR / CnMpljTFk RFSFARCH CFNTFR ) 

nnniFiFo 4- NrtvFMRFP i.q7«5 RY/'m. icaofn. 
MnniFTFo ;>9-ncT6RFP* iq-ff* Ry n/vin colfman 

. . . ■ . ; r 
*****f;pNFRAL:' . ^ ' 

ou'rSTinNS ANo commfnVs SH^iii n. RF oiRPcrPH rn: 

SUPPORT STAFF MANAaFP 
. COMPIITFR RFSFAR^.H CFNT&? Rpft . FCONQ^I ICS AND ANAGF^FN T SOTFNCF 
NATIONAL RURftAir flF FCONf^MJC RFSFARCH " 
^75 TfrCHNnLO^VY SOMARF 
CAMRRinGFi MASS. n?i^cj^ • , 

• ' ' ■ - * * ^ ■ 

OFVFLOPMfNT-nF THI5 PPPGRA^ Jl^PPORTFD IN. PAP T KY 
MATTONAL /SCTFNQF FOllMO/i T ION ANT GJ-l 1^4X3 ANO . 

NATIONAL SCTFNCF FnilNDAT ION- C-frANT nCP7S-nRP07 . . 



RlWOnS^sO* 
HIwons7n 

- RIWOnSRO 
RI WrAOSQn 

RTwoof^on 
Kiwon-tjtio 

RIuon<y3n 

RT.Wn064'0- 

Hiwnn^^o 

RTWOn<S6.0' 

HIwon^7n 

RIwnOiSR-O 

BI wnosoo 

. RIWr«07nn 
RIW007in 

RI won7?n' 

Rlw?Vo73o 

HlW'>074O 

RlW^MVyftO 
Riwoo77n 

RTW007Rf> 

J3iW007Qn 

R.IW'lORin 
RlWO0M?f> 
HIWO0R3O 

. RiiyooR4n 
; BiwooRsn 

RIWnOP60 
. BIWn0^70 
R IWODRPf* 

RIWOOOOO. 
Biwoo<?in^ 
RIWOOOOO 
BTWO0Q3n 

Riw»vno40 
BiWnoQSn 

RIUOOQ^n 

RTWongjb 
RlunnQRO 

8IW:>hQQf> 

Riwnxooo 
Rtw'Unin 
RTWoi02n 
. RI wnin3o' 
. Biwni040 

• BTwnj[ (^f,r) 
6 T wni07n 

BTW0.1 fvRO 
KfwOlOQO 
RTWOllOO 
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FP.Ps AIWGT ^ FORTRAN >^ 
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in MAJ/tCTNAL ftHRFAU OF FCONifiMJC RPSFARCW, INC.. 



:::::: s : 1 1 ::::::: :r r :::::::::::::::::::: ::: : j^: ::::::::::r^T::ir 
r 5 : 2 : : : : : :Ti : : : : : : : : : : : ::::::::::: : : 

' nFLTH -IS TWF LAftGFST PnsirjvF Ft.OA riMr;. phimt, nu^rfr' 

TF (TRMl) TWFN . '^ : . . . - Z^' 

-IBM 36n/37nr npi. Im . ^ ( if> . **63 ) * < 1 • - IS**'*-^) rr::::'::M 

Fi,s? IF .{ rBM? J .'tmfm ; 1: . • : ■ .;■ 

/TBM 370/3^n: * OFLTM = ( 1ft . **63 ) «c( 1 . - X6.=»«-TA) ::::::: ^: : 
FLSF IF (VERnX) .THFM' ■ 

XEROX: OFLTM = ( 1^,, <c*f>.3} * M ♦ - 16,*=ec-6) ::::r:tr:: 
F.L^F IF (UNTVAC) THFw .. ' ' . 

\/UNiVAC: HFlT*^ = (;7,*<ci27>*fl, - ^•*=»-?7) r :::::: r: : 
.ELJsF IF (MIS) THPM • , 

. wONFYWff LL:/0FLI« ^ ( ?.<c<cX2? )*M. - ?,*<c-?7) 
FLSF TF (bEC/ thfn" . * 

POP 10: OFLIW = f ^•=»*i;?7.)*(l. - ?.m^^-?.7) ,::':r:::::: ■ 
FI.SF TF CCbC) THFN - . 

CONTRPL DATA: OFLrw = "^(p.xxri'o;?;? j-esf:?^<:j6r^R -1.) ::::::: M : 
ELSE- IF (RGH). THFN ; . . " ^ 

RURJ^niJf^HS: OFL IM =* Yr-.«^ 1 .»::":::*?::: : 

. ELSF^ I CARO ' > • ' ; . - . 

iQe4e4e«:et4i4e4i4e« DATA STATPMFrOT **<c**i*r^*** 

PAT& DFLTM /STNFP/ \ 

DATA OPLIM /77FEFFFF^FFFFFFFF^/ ^ . 

::::::ti:: , nFFTA.-TS tmf Smau FST pnsT ri ve flhattng point -NiuMBFR ' 
SVT^ OFETa JiNn,-|I^FTA C/^N- f'OJ.H RF .WEPRESFWTFn. 
-I-F (IBM). tHE^l■v■*^:^^;f-C;;;;: ■ --^-rjlsv:^' -\ ... • - 

- IBM 3^0/^70:" !!FFTa;''= 16 •,*^-5S5";';;: ■ 
ELSF TF (XEROX ) THFW ' " ' • ' 'V 

XEROX: UFFTA. 16. **-AS 
ELSF- IF ^fUNIVAC") - THFiyi ^ 

JiWTVAC: llF.FTA -= ?.**-i2c : : : i i : : : : ; 
ELSF IF -(wrS) THFN / /• *" 

HONFYWELL: UFrTA = ( 2 **-?7) . : i*: : : : : r: : ' 

EL.SF If .( OEC V THFIM ' ' 

POP inr (iFFTA = ?.<c*-12t :. - ' V . 

FL^^E IF. (COO THFN 

CONTROL nAja: uffta = 2.**-q7^ .-::::::::: : 
FL.SF- TF fRGH) THFM ' , . , " . 

RU»-RnuGHSt IIFFTA p.*<c-S} :::::::::: * 
ELSP, 1 CARn 

>4e:^4c4e<t4e:Cet(« O A T A S T A T F MFN T sf <e :}! j}:" . 

OATA iJFETA /SETA/ v • ; - ■ " 

DA.T A iiFFTA . / 7*001 nnononononf-rr/ -. . ' . 



<ca}c*<c*RnDY OF PROGRAM*: . , / 

TF ,(CnwST ^LF.', UOOn.) PR:nO ; = *f.FL,TM * CHI^ST > 

IF (CONST ;aT\. I,.0b0) PffOO = MFFTA CONS 
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A Ks t ffac t ■ : ■ * . \* • ■ . 

The paper describes an * experimental 
interactive system for the solution of 
Integer programming problems (prfmarrty of 
0-1 nat^re^ but, not exclusively so) - 

The ..system I riclu^'des^most techniques, which . 
appear to of fer hope ; -of overcbmlnrj the - 
combinatorial ■ dlffiSuKres of al l Integer 
programming algorithms: In particular LP 
with cuts;!r 88 w.lth propagation^ State- 
enumeration^ Jnterval reductiony Preferred- 
^'^.rfabie*^ reduction^ Exploitation- of 
deVs Iniequalltles^ LocaT %earch^ 
r 1st Ics^ etc. • . 

There I's emphasl-s. on Interaction by_ . the 
analyst at the terminal. An example 
Illustrates'^ the . use of varl.qus tools. A 
table summa r I z<?s resul ts pbtarlned by a 
number of dlffeireat' approaches to' a 
problem of moderate.^-di f f Icul ty.. 

11 ' Introduction - , 

V/e describe an experimental 
Interactive syst^em for* 0-1 programming. 
The programs are wrltJren in APL. 
-Various, techniques have been or are 
planned to be * Incorporated In- this 
system,. They are Intended to be called^ 
By thjB user^ rndependen<tTy from eac6 
other (subject to restrictions which vre 
. would like to make as little painful as 
possible) , or sequentially; depending 
on the results of the experimental I on:< 

No single technique can be 
expectejd to solve all types of Integer 
- problems. But our preTlmlnary 
. experimental resul ts are encouraging 
and suggest that . a \ truly flexible 
Interactive system has potential for 
aiding In the understanding arid"' 
sblutlon of. Integer programs^ ^beyond . 
wh^t can be expected . from a standard 
preset program. ' ^ r ' , 

' ^ The paper Is not In tended ^to give 
all algorithmic details. In Sj.ectron 2 
. an overview^ of techniques and features 
Is g^lven^ with references to expository 
papers* For convenience^ certain key 
concepts are briefly summarized In^ an 



appendix; 

What 
ar<g certa'ln" 
at . the 
loformat Lon 
Ce.gX Jn 
relations 



counts here Is* that there 
techniques for* /'looking" 
problem, gleaning more 
from It In simple form 
terms of simple logical.* 
among variables/ simple 



^neodal It les, stronger bounds, etc.) by 

termlnaT . gti I ded by the printout; of the 
.HciiLCfinl!! state of- the solution 

, process. ^, The user may find ^ome'help 
from the example presented {n section 
3. ^ \' . 

Can Interactivity play a -major 
rol e In solving difficult problems? We 
increasingly believe that the answer Is 
yes If the- user of the system Is well 

^versed. In Integer programming* Whether 
one shall be able eventually, 'to 
construct usefuf interactive system for 
the / n^-spec la-l I^t Is - somewhat 
difficult- to predict *riow. We are 
optimistic, but much work remains to be . 
■done, . ■ ■ ' ' . 



2. Techniques of the Svstf*p 

^ At presen^t the fol lowing features 
and techniques have* been Incorporated : 

2^1 Linear Programming^ (LP)^ 

v/Itb possible addition of 
Gomory-Johnson cuts ^. and dual 
reoptlmlzat Ion until either no real 
progress Is made any more Clh terms 
of ch^anglng objective function), ' or 
^untll, the number of reoptlmizatlons 
" . or CLNts reaches upper bounds • Imposed 
by the user. <l,2,3,-i>> 

2-2 ^inajach and* Bound Profframmrnp- f , ^ 

. with oropagat Ion ' on oonbasic 
(possibly - jinafsJinad.) : varlatrt.^. A 
V single LP Is solved at the. current 
' .origin (or node) of the search tree, 
and raenal t leg are computed. A branch 
. on a nonbasic variable set at Its 
optimal LP rvalue Is called 

oropagat Inyr , One will want to 
propagate as' long as. the "al tf*rnat;f*'' 



an 
be 



Penalty Is large^ <:5^6>7>. , 
l^ij^ Bende rs -\ neoua 1 it ? . ; 

i-^ which are .seneraffted at the end of 

- .'-^ . e-xecuted LP. They ^re Intended to 

*used eFther^- , ; - • ^-r"""., - 
• ("I ) - after . solution *'vof, a prob7 

within a ^ . BB: or Enumeration 
scheme ' {usually ^ vsti^h sorne aid 
from cutting; plam?s>V or 
. . (i i) after guesses - \ai! partial 
^ Integer 'solutions have been 

entered by the user ;In order to 
~ \ '^et- logical conditions which 
* v^. ' might characterize the problem. 
^ After a guess Is entered^ the 
remaining T inea r program Is 
resolved and the Benders 

- •■ Inequ,qMty Is * taken from" the 
. ' * ^Inal LP tableau Cwhether 

feasible or Infeaslble). 

'\ Note: It may not be permi«lble 

to use cuts In such an Instance. 

In general^ Benders Inequalities are 
collected and re t a ln>> ff for 

exploitation In terms of reduction 
(sees 2. >^ 2.5 ^ below). In contrast^ 
Gomory- Johnson cuts (and ive believe 
other cuts of similar genesis as 
well) are not well suited fo> 
reduction (i.e. ^ they lead to logical 
Inequal Itles of large - degree)^ After 
use In an LP, they are therefore 
•usual ly discarded. <8>9> 

Z.k Reduction * Procedures - 

are invoked throughout to ■ r, 
• (I) shrink, bound Intervals \^ and ; 
^ ; . ( H ) to generate logical mequal It les 

.■; -v."; '^"^ A - . . ' 

... -The. Interval reduction procedtrre .Is 
- used. Itera'tlvelyi, screening a.1 1 

v. - . constraints so as ten compute the. 
tightest possible V bounds on 
. structural and slack variables. .Great 
1 care must be taken to avoid roupd-off 
difficulties; i.e..>. one must use 
appropriate scaling and tolerances 
(as In iP)^;^especially when Benders 
inequal F tiles are used for reduction, 

2.5 Lo;;lcal Inequal Itles ^ . » 

■ * 

• ■ . - ^ 

Given any Inequality in (0^1) or 
Integer variables^ one can derive 
from It a set (possibly empty^ but 
. not usually^ "so) of min Imal oref&rrf^d 
variable inequal Itles (m>p. I . *s) . The 
4^t.r.^^. of ^he .system ("size" of the * 
smal lest Inequal It les; the degree of 
an Inequality i will be called f^^Ki) 
In the printouts) Is a good measure 
of the tightness, of the problem. The 
bound Interval of one of the 
preferred variables^ at 1 eas t^ \cpust 
be. shrunk by one unjt. 



These logical Inequal I t lies ' are among 
the main tool-s for guTdlng both the 
. B8 scheme; and the ^^.enumerat Ion / (see" 
also V "penalty " Tmprovemelnt") . 
<9,10.1i*>15> . - 

-As • the referee points out^ .^there 
ejcIstV logical inequalities 

("Boolean/^ "jCanonlcal" Inequalities; 
e.g.^ seie <16^17>) which are more 
general than m'.p.l.-'s.- They are 
- especially Important In the complete 
characterization of the underlying" 
problem. However^ their overabundance 
. V -present - 'computat I.onal' 

dff f Icujlf^les.tJite believe we have good 
reasorfs. for 'preferring to work -.with 
^ the more \^jJ>ec\al prof>ertles/ .of 
m.pir. 's "^{used computationally 
already In <9>). 

2^ gnun^ ra t Ion . .. . . 

IS based on the addltlvA al gor I thm nf 
^ . Balas <18>, modified In a number of 
» ways^ with the search starting at the 
origin y«(0,0^...,0). Branches are 
restricted to mlnTmai Jix&fsJX&d 
variflbT^ i£i5..' The- actual set'^to be 
used- Is the ^aoLqii -of those preferred 
variables (in minimal preferred sets) - 



good :cQntr^r1^i9n 

double contraction 
single contraction.^ " 
(defaults :X al 1 Ofe^ 



which . ''have 

1 St priority; 

2 nd priority: 
. 3 rd prior ity 
. varlabl es . 

(a branch . is "contracting" when the 
degree . * of the system-Is guaranteed 
to be decreased as a. result. 
Contraction can be determined 'from 
' this, set of minimal 'preferred 
. inequalities). Within • the finally 
"chosen set^ the actual branch 
variable Is selected ^o as .to lead to 
minimal ^ (maximal) ' overall 
Ipfeas ibl l'lty of the , problem at the 
next node In phase 1 (phase 2) of the 
enumeration. <10> 



In the above context^ phase 1 Is 
meant to be the period diiring wh^cK^ 

d I reacted towards" 
feas Ibl e sol ut Ions ^ 
Is to deal with the 
opt imallty (poss Ibl y 
tolerances) once a* 
solution has been 



the search Is 
finding improved 
whereas phase 2 
establ Ishment of 
wi thin certain 
good Integer 
generated. 



f 



Note:. These branching criteria can 
^ rendered Inoperative when 
the "loca/i search" (see ^.10) 
\ Identlflyes an irm>rovIrfg 

directfdn. * " x 

< State Enum<^rWt ion. * f. 

differs from enumeratSiQn as follows.' 
. At any no.de^ a state Is determine.i^ 
(usually by rounding of an- LP 



solcitlon}. It \s essentially a 
partlcuFar value for the -integer 
vector \\y, say : y> bellevcfj to be 
close to'/feas*? ble rnt^^'er/solutloos; 
. In the"^ zi^vTO-one ; case^ the/- search 
variables for an 'enurneratron/such as 
that of 2.6^ are then taken to be 
y(j) If y^-) « a and 1^- y{j ) If 
y(j) - 1. (See <12'> for the 
iseneral Izat Ion to the Integer case 
i^ith small bound Intervals). 

"I'.e.v * .the final' enumeration 
carried " ,out . with' transforrned 
va r I ab 1 es y ( j ) set at .0 Initially 
^and Increased on forward steps of the 



algorithm Ca's In *<18>). *\n 



State Enumeration^ we can envisage 
that the search variables y(j ) may 
be redef I*ned . over the free/variables 
at each ,nt3de/ provided that 
reasonably good Information ^s ^ 
available for doing so. 

V/e have found State Enumeration^ wit'h 
s trateg Fes roughl y as* outlined In 
2.6^ 2.7 , (there are " many other- 
possibilities of sometimes" highly 
--""Complex nature)^ highly effective In 
finding ^^^^.iiC^ feasibly '' Integer 
s'olutlons f^Sicv^ especially when 
augmented by b simpJe "one-lever'V 
local search* any by a "c<i>? 1 tg*st" ' • 

, «• (see <2,8/2.9>)L <S3y'*^l'*> 

V Most enumeration schemes have tests , 
Invol ving the obj ect i ve- f unct I on of 
the initial > tableau* (often' . .with 
J non-negative cost, coefficients). In' ' 
. state .enumeration^ the original 
objective function becomes less 
interesting anc! dan be taken care of, 
anyway, as a. special Benders 
Inequal it^. Instead of the original 
objective function, we have found. It 
useful to concentrate on the final 
objective. function row from the * 
optimal* tableau of an Initial LR 
problem (with cuts added .when deemed 
. ^ desirable). -The row is retained 
(coefficients and all -necessary 
information about the nature of the 
nonbasic variables) and permits the 
fixing of variables and/or shrinking 
of bound in terva Is for the *nonbas Ic 
variables, among them original slack 
variables as .well as structural 
, va riables . 

It should be noted that the shrinking 
of slack hound intervals can be used 
as Input to the various reduction 
procedures and can lead to further 
Information about alithe variables 
there. , ; ' - 

^. 9 > Bounds on Slacks ; Compression 

*• ■ ■ ■ " v 
Our linear prograrming system and alV'^ 
related Integer progi^^jnmlng 



prorcedures permit the imposition of 
upper and lower bounds on the slacks. 
Equality constraints, for example, 
• are h^iandled . b> imposition' of zero, 
bounds above'and below. The miniM.al 
Inequa 1 1 ty reduction procedures work 
with inequairtles only, and we 
generate two suchV inequalities ass 
inpXJt to re'duct ion" whenever we know 
(or believe; that the upper bounds- on 
the slacks are ' "true" (I.e., truly 
restrictive) bounds. 

Some of our sample problems are known 
to have two inequalities representing 
actually only one equality 

constraint^ The system therefore ,has 
a "compress" . function detecting such, 
situations , and (re)creatln£ 
(or Iglnal ) equal I ty constra Ints, 



the 



2,10 Logal Sf^arrh 



Any, 5B pr Enumeration algorithm niay, 
benefit from a search around' the 
/'current" point under^ 5Cohs'ide rat Ion . 
In^the present system^, the "depth" o^ 
^ the local search Is controHed by , one 
parameter ('LEV, or % ). The value Q 
; corresponds to ho jBearch '( j ust one , 
the current, point considered), the 
value 1 corresponds to ' the 

a'lteratlon of all ~ f "free" (not 
fixed) variables by one , unit at a. 
.time.. In general, one enumerates the 
' ( f ! 0/ (f-i) ! adj acent points. In view 
of. the exponential Increase in the 
^number of such points, only the 
-values 1 'and 2 and conce'ivabl y 3 
appea-r to be reasonably for . ^ 
- ■ ■ , ■'. - . * ' ■ ■ ■ ' 

2.11 l^urlstics for Feas t bTf^' ^ Sol ut Ions 

Severa 1- he'ur Istlc^ methods Kave been 
. programmed which* try to generate 
feasible' solutloiis to a system . of 
T Inear Inequal l^Ies in 0-1 varrables. 
• One of them starts with a given, Q-1 
vectoi* and modifies -the^ component 
which maxima 11 y decreas.es the^^j^m of 
. ; the Infeasibll I ties or the numfedTr of 
lofeas rbl I Ttfes. . Such steps are " 
repeated as often as specified. One 
way of using. such, modifications is to. 
have a complete "forward pass" over 
all fhe variables, followed by a 
compl ete "backward pass"^ I.e. alj 
variables are altered in a pass,, the 
sequence ibeing determined bY"* one of 
the criteria mentioned above, . <19>.. 

Another heui^istlc concentrates -oh the 
lo,gI^a.l inequalities oT' small degree 
and tries to generate a feasible 
solution to the "current system of 
min Ifnal 'preferred Tnequal I t les, which 
^ can then be used as a starting point 
for the first heuristic, or for a 
local search, ..or 'as a- state for the 
^state enumeration algorithm* <3,U^JU> 
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2.12 novi Comb irtat Ten 



it has been known for a long while 
that linear combinations of rows may 
be more Interesting "than the rows 
themselves; but there Is little 
Insight into how the inequalities are 
to be generated (i?ossible^ except Ions 
.are references <20> and <21>). 
Consistent wfth-the main emphasis of 
this paper, i/F? - have chosen to take 
the ^ degree of an inequality as Its 
basic measure of strength. Hepce we 
al low for "visual" comb inaction of 
C ^ ) Inequalities (e.g.-, so as to 
have coefficients of opposite sign 
conblne to produce sra 11 Tesul ts in 
magnitude Von the' left' and larjce 
negative right hand. sides, * if 
poss ibl e), and we test the ^-esultant 
inequality by reduction. 

Alternatively, we also have heuristic 
algorithms, of modest capability which 
utilize ininimal preferred 

inequalities to give, automatic row 



An Examplf* 2£ 2. Te rmlnal Sessiog 
1) STARTl 



BM1515 Is a 
the form 



Co 



(2:6 by 1.6} tableau^ of 



combinations, -with reduction used 
again to <:^heck but the results and 
terminal?^ the process. 

♦ - s 

iU2 >mnrov<^mf?nt q± Pena? ties 

^' Penalties can be improved by takijig 
t -into account lojcical conditions on 
the integer variables, such 'as the 
minimal oreferrec^ inequalities. If 
STome nonbasic variable must, be 
modified It may the case that at 
least some of a set of other nonbasic 
variables must also be altered to 
achieve feasIbUIty, This usually 
leads to the accrual of an additional^ 
penalty. One nay utilize improved 
^ . penal ties <a. priori In a preprocessing 
of the penalty table for 
pro.^crammin;;, 6^- dvnamigaM v to store 
no^es with Increased alternate 
-penalties in BB with propa-ga t ion . 
<6^7> 



corresponding to the minimization of 
z « Co + C . y, subject to: ^ 



A . y B 

O^y^l.all y(j) integer, . 

It represents a 15 variable problem 
of /moderate difficulty, with. 15 
constraints and a dense constraint 
matrix (taken from <22>, problem 
*13). 



STARTl 
ENTER M^n 

' 1^ 15 f 
ENTER TABLEAU^ EXPANDE D FOR K 

0:* -: - . . . 

I>3A/151S . . " 



0 

*36 
22 

"10 
"12 
1 
2 
3 
"2 
't^ 
"9 
25 

'32 



7 
2 

"is 

5 
"9 
8 
7 
"l 
*1* 

"1 

, 8' 
*2 

7 

7 



4 

5 

5 
"5 - 
"5 
"7 

5 

3 

2 

- "6 
0 
3 
0 

'9 - 

. 1 



3 

0 
"3 
"3 

9 

5 
•0 

i+ 
"9, 
"2 

0 - 
"1+ 

0 
•6 

"3 



4 

3^ 

0 
^8 
"1 

9 * 

6 - 

1 

0 
"2 

0 

S 

2 
"2 

1. 

0 ♦ 



2 
3 
'^X 
8 
8 

6 
0 
7 
0 
0 
5 
6 
"5 
■"5 
.)5 



6 

2 
'3 
- 9 
■"3 

7 

7, 

1+ 
"9 
"6 

a 

7 

7 . 
"9 
'5 
"3 



2 
6 

"a 

*"2 
9 
1 
6 

"l 
9 

"6 
0 
8 
1 

"2 
»l 
7 



1- 

2 
"9 

0 

9 
"3 
^7 
'.""6 
' .6 
. 7 
,0 
*8 
""l 

2 

9 

8 



5 
2 
3 
9 
3 
•2 
"7 
0 

■ 

""l' 

0' 
1 
^8 
3 

.1 



1 

5 
"8 
"l 

. 0 
* 0 

5 

0 - 
"1 
,0 
1+ 
1 

"6 



3 
6 
7 
*"7 
"3 
"8 
0 
3 
2 
"l 
'l 
"5 
■"3 
0 
6 



7 

3 
"3 

9 

5 
"5 
"3' 



8 
■"l 
"2 
"7 

5 
0 
8 



7 
9 

if 
"9 
7 

i; ' 
■ 

"9. ■ 

1 . . 
"6 

2 ' 
'8 
"7 

8- 
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2} rhitfal LP. Preprocessing, 



No var fable can. be ffxed by 
reductlpn- In the initial 

preprocessing. The contlnous. LP 
optimum IS lif.95 and^ 5 variables 
out of 15 are fractional. A Benders 
Inequality Is computed from the 
optimal -LP tableau and printed out,. 
(ZSTAR stands for the best objective 



function value founds or for. 
upper bound supplied by the^user. We 
use a large value as the default). 

One enters a "state" STl, 
usTnz a rounded LP -r-Ssal ut i on wl th 
rounding parameter p ■ .5 ^ (entries 
l-.keep yCJ)^" entries 0..use 
compl emented variables 1 y(j )■) / 



BBIJDERS INEQUALITY OF TYPE 1 
(5*844 0 '6.122 6.7U1 3.03^0 0 
1.95T "0.7153 ) X y 5 "25 



STX^l 0 0 11110 11 
Ik 




10 0 0.7512 2.862 8.055 



3) nup<;y;f»5; 

One Is prompted for guesses. 
One enters Indices and corresponding 
• values for a subset of the Inte^^er 
variables (unspecified- variables 
being left unconstrained) . and 
^resolves resul t Ins LP. 

ihp final LP tableau yields a 
Benders Inequality, of tyc^ 1 if 
feasible^ of type 2 If Infeasible. 

The function- GUE'SS works th the 
last BenderTs Inequality* 

GUESS 0 chooses the fr&e variables 
with ne^a_t^I_ye coefficlenrs and sets 
them to 0. The j^uess Is then llkeTy 
to be Infeaslble and a new 
feasTbilltV condition >;"n be 
generated I . of type 2). 

. GUESS 1 chooses the >f ree varl3b*les 
. ' wi th pos 1 1 Ive coefficients; and o^*?' 
sets these variables to 1. 'In this 
case also one constrains the current 
B.I. and tries to obtain tighter ' 
feas i bl 1 Ity condi t Ions on a subset 
_ of t.he variables. - . . 



Any given j?uess leads to the 
resolution of one linear' program. 
While the Benders inequal I ty i s* 
-retaTnied for . fur the r process fng 
(possibly after a check as fb" 
whether It Is strong enough^ I.e. 
has degree low enough). For large 
problems^ the guesses will most 
likely be generated automatically ^ 
according to a scheme which the user 
may have developed . for a smaller 
problem of similar structure. 

Having some fami 1 Tar ity wl t h a 
given problem (and by that we do not 
mean "^noWTng Its solution)^ one can 
usually think of a number of other 
promising guesses^ £.g.^. one may 
want to- set to one. some . variable(s^ 
with large' pos It Ive coef f lent (s ) €10 
at zero some varlable(s) with large 
negative- coeff Ic I entsj or one makes, 
guesses In consonance with one's 
knowledge of any possible special 
structure. 



WAIIT TO GUESS AX SOLUTION? 
YBS " ■ ■ - . 

ENTEP GUESS' FOR r." FIRST INDICES, THE?^ VALUES 

n; . - - . . ^ ; ■ • - 

, ■ "GUESS 1 
. 2 "ro 12 ■ . ' x - 

0: .-■ . • . ■ • ^ 

/ - , - . • . ■ 

. ■ ■ 1 ■ . . ■ V 

actual: nUMBER :PF RQWS FOR MAT :18 

FR0BLE14 NOZ FEASIBLE 
'DETERMINANT- l.dOhES ■ " 
^BENDERS INEQf^ALlTY OF TYPE 2' ■ " - "'^ 

C 39: "^1.563 "10. 1-1 0 11.02 5^7^ 0 O "6.753 2 5 . 6 3 vO^/l^fS 0 0- 0 ) 

. X 7 ^ 12.12- . : _ " ^ ^" : 



WANT TO GUESS AT: SOLUTION? , . 

YES ^ 

. EJiTBIi GUESS FOR I ^ FIRST ' IlfDlCES ^ THEN VALUES 
Hz 

.. ■;. ' ■•■■7 . ■ : 

0 . ■■ 



11.^7 SEC . 
OBJECTIVE FUNCTION = 15.12 
iro. OF PIVOT STEPS = 29 
DETERMINANT= 1 . 045^6 

STRUCTURALS: 0.01126 0*7627 10 0 0.087U 0 1 0.3813 0.2G13 0.1264 

0 0 0 1 - 
BENDERS INEQUALITY OF TYPE 1 

(00 "^9.671 8.708 2. 2S6 0 "3.909 "0.66U5 0 0 0 .3.872 ll,2i 2.857 ' 
6.548 ) X Y S ~'22^ZSTAR . . ^ 



'^) After all guesses have been entered^ 
and the corresponding; LP's- have been 
solved^ with a final ftenders 
fnequalTiy always adjoined to ^e 
tableau^ PREPROC checks for^Tnterval 



reduct \ on and f T x I nj; "of y a r f a b 1 es . 
<ln this.Aroblem variables yl^ y9 and 
yl5 can be fixed at 0, 1 and 1 ^ 
• respectively^ 



N. VARS. FIXED 3 



0 - 0 0 

oil 



-7;£A2C1;] 
0 jO 0 0 
LU2L2il 

r-1 1 1- 



5) Trie starting LP for t?he BB (or an 
. Enamerat;Ion) ,pr5>cedure Is"^"^ then 
solved _wrth^ these values 
substituted^' and the Initial 
objective function value Is thus 
raided to 15.7 . 



-v'l^C*" bounds') 
0. 1 -0:-0/0 0 0 1 

1 '1. 1 1 .1 1, '1 1 / 



Dual reoptlmlzatlon after the 
generation and temporary addition of 
Gomor^Johnsbn cuts 'finally brings 
the Initial BB objective function to 



OBJECTIVE FWfCTION - 15. '7 ^ 
NO. 'OF PIVOT STEPS = 25 
DETERMINANT^^ . 4.987^5 

STRUCTURALSz^O 0.8451 1 0 0 0.1926 0.1758 0.3951 1 0.2186 0 0 0 0 

■ l" • " 

BENDERS INEQUALITY OF TYPE 1 

( 7.362 0 "4.894 5.722 1.515 0 0 0 3.848 0 0 1 • 463 8 . 02 1 .3 . 042 
1.645 ) X y s "15.1 -^ZSTAR 
5 aUT(S) GENERATED 



OBJECTIVE FUNCTION = 18.39 ■ . ' ' , 

NO,. OF PIVOT STEPS = 87 . ' - 1 

BENDERS INEQUALITY OF^ TYPE 1 . ''^ x ' 

i i?.^^^ ^ "3.911 3.535 0.09996 -0^.08115 0 0 4.343'-'o 0 0 5.384 
-0.004794 1.39 ) ^ r s; 16.57 ^ZSTAR ^ 
STRUCTURALS t-O 0.1975 1 0^0.0 0/6106 0..4917 1^0.185% 0.101 0.5849 



6) IB. StartSi. The system "computes' 
penalties (added to the current 
objective function value to give a 
maxTmaJ lower bound on the final 
objective function value). 

In the normal, BB ' mode^ the 
generated node <|.e.^ the current 
bounds and.z ♦ penalty) Is stored In 
the node table;^'^^d the ne;jct node 
would be brougge^ln. 

• n the propagation jffijds. one looks 
for variables to be fixed at their 
current value^ with an alternatf* 
nadfi stored In the table^ and the 
current nstds, being processed 
further. 

- . • ■' , ^ ' 

In this example a propagating: 
variable yCl3) Is Identified and 



the alternate (y(15)«l; .often other 
conditions can be Imposed from ".the 
minimal preferred Inequal ItTes) Is 
stored with penalty 5^384. 



The 



tbe PI 
noft • t^ 



user Is asked whether he accepts 
propagation. If the answer Is 
the propagation proceeds; If 
:he user has the . choice : of 
proceeding In the normal BB mode or 
of trying to finish the. current 
problem (In core) by state 
enumeration. In the present case we 
accept the first propagation 
(storing an alternate node with 
pe^nalty 5.33^*) and reject the second 
posslbll Ity with alternate penalty 
3.911 (the system^ of course^ 
generates the penalties , In 
descending order of magnitude). 



ACCEPT 



PBK Aury rf ^ ; 

OUT:. ; Csr^ /vci>e) ^ ' . " : " 



PnP^PEN.CTR\ Z ^11 -0 _ ' 

ACCEPT ? 
NO 

FOR ALTERNATE CALL STRAT^ BP, ^JBP ^RERE'' 

' SAVE NOW. BEFORE ENUM. 



7) 'EnufTTer.itIon star^$ (from the state 
STl). Variable 13 Is at 0 (via 
propagccion)^ and yl^ y9^ yl3 have 
been fixed globally. One branches on 



y8«l, and a feasible solution Is 
found Immed la tel y (undoubtedly due 
to the Imposition of the state)^ 
with obj function z-26. - 



PI A ^ A 2 2 If 3 2 3 10 5 3 *f 3 10 ^ 2 2 2 10 10 2 10 3 2 10 Wrv, v J \ 
K IS X C\ec-.l s*e.«k level) 10 2,10.3 2 1 0 S (f>:vi dcg^tSlC^Cj . 

Z 2^ • ... ''^ * . 

FEASIBLE SOLUTION 2S ' * ' . 

SOLUTION FOUND AT' NODE 2 LEVEL 1 PRASE 1 

PBJECTIVE FUNCTION 26 . ^ ■ ' • ' ' - 

NEU ZSTAR =26 ■ . ^ V ^ ' . . . ■ * 

SOLUTION 0 1 1 0 0 0 1 1 1 .0 0 1 0 0 1 

-INT. SOL. FEASIBLE , 2= 26 ' . - . 



8) 



Enumeration Is completed after 



examination of a total of 17 nodes. 



l/BND= ^l . INF. INB0UI7DRD C/N ^UNJ> K^2^ocrtcN\ 

END OF ENUMERATION ' ■ . \, 

\ OPTIMAL ' SOLUTION. 6 1 1 0 0 0 1 1 1 0 0 1 0 0 1-^ 

OPT. .CB J, FUNCTION 2 B • > ' 
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9) jjsiiim isi M» 

. The- stored alternate node Is brou,<;ht 



from the tree into high speed 
storage. After [.niposi t ion ■ of the 
bounds, _ the 



w»-»tjiiuai,_ cue inequality system' 
(including Benders 



of 

, . Si. 
I nequal it ies } 



with z £ . 26-1 

infeasible . End of BB. 



» 2 5, is now 



Tern tnat I on of Integer pro^^ram. 



in OF IWDK 1 



1 23,77 5,384 "13 (rtoDE NUMtSef^" V^Lue ,?eN.^Bf2.VvaO 



RED . ^^R'fPUCTioH; 
NOPV^ 0. CjC or 3yST£M). 
P?L:/5 '20 15 ' ■. 
I'1:FEA3IBLEi:J0P.V . : .0.* . . 



iLt Comouta t ional Rgsul t<; 



There are a number of problems 
which can be entirely resolved by the 
use of one or the other technique. For 
example/ a C28y.35) problem of - the, 
tanker, schedul Ing variety (see <9>) 
yields an* integer solution after 
impos 1 1 ion' ' of. ;a total of 6 
Gomory-Johnson V-^ cuts in 2 

reopt imiza t ions . 

The -same problem yields wel 1 to 
state . enume rat ion techniques (11 BB 
nodes^ of the propagation type^ i,e. 
with only one : true linear projjram 
resolved^ : f 61 lowed by between 9 and 33 
^enumeration nodes depending on the 
Initial SLtate;\at thie end -none of the 
altepnate SB nodes need to be processed 
further). * - ^ - ^ 

On* the other hand itrequires a 
large number of LP optimizations In a 
straightforward BB optimization run 
(between , AO and 109 Njnear. programs 
depending on strategy)* " 

; In the following table we describe 
various^ approaches, to the (15 by 15) 
problem used, as example In the text. In 
some respects 1 1 - Is rather <Ji f f IcuTt : 
the gap between* LP ob j . function and 
I*nteger solution objective function Is 
large; the penalties are rather small ; 
the response to Gomory-Johnson cuts is, 
only moderate; the degree of the 
Initial system Is 3. behaves much 

better when Benders . Inequalities are 
added^ permitting the reduction ^of the 
degree to 1 XfixTng of three 
variables); etc.. 

In the table we summarize a numblar 
of runs using some or all of the * ' 



available tools. One does or does not 
add Gomory-Johnson cuts. One does or 
does not enumerate at ' a selected 
pending node of- the BB tree. The state 
for enumeration can .be chose^ 
arbitrarily:- we' ran the proijram wi tfr\ 
the state determined from the rounded 
LP solution, with "all fractional 
variables less than or eqt^T to '^5 
fixed at 0, and a variables larger 
than .5 fixed at 1. . 

We entered some « guesses at a 
solution and appended the resulting 
Benders Inequalities* to the. original 
tabl eau. . 

Which of the resul ts ar'e 
cons irfe red to be best w71 1 depend on 
the relative computational efforts, and 
therefore^ on the computer and on the 
system used^:(APL In our case). In- 
general, we feel -It most essential to 
reduce the number ■ of linear programs, 
but even more f unriamenta 1 1 y,* to avoid 
^arge trees (many .BB nodes, large . 
enumeration levels)-, i .e. s I tuat I ons I n 
which', the combinatorics overwhelms the 
problem solver. 

The interactive system . appears to 
give the Ins^Fght and often the tools 
for avoiding exponentially Intractable 
si tuat I ons; 4 i;e., even 'for more 
difficult problems (e.g., the/(37 by 71* 
problem of <9>), which we dldl not run 
to ^conclusion because* of slow.' APL 
response at this terminal, Tt was. clear 
that the search was "well --beh^ed", the 
tree remafnlng a "tow-level tree"., 
with steady progress, be Ing made . 
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Initial LP val ue 

- . Total of cuts 

reoptlmlzat Ions 
Final value of LP _^ 

Integer optimum . ' • • 

Use B-5? 

N. of nodes ?;enerated 
•'•••«•••••'• ca 1 1 s d , 

Optimum .found at node 

'.Use enumeration? 

' State for enumeration 

from LP?. 

1? 

0? 

" ^N. of nodes 

Optimum found at node 

Use. guesses? 
' Type of guesses 
N. of K^uesses 
fi» of. Benders ineq. 
N.' of variables fixed 



1U.95 14. 9G 14.96 14.95 14.05 

r ' ' I » . » . ! t • ^ 

: 0 ' 0 0 47 ! 35 j 41 

0 ■ 0 / 0 7 : 7 ; V 7 

14.96 ; 14.95 14.96 10.53 : 15.53 13.30 



26 
no 



yes 



yes ' 

105 
73 



yes 
13 

11 • 

12 . 

no ; 



no 



no f 



26! 



26 ; 



N. of LP solved 



yes 

2 
2 



yes 
yes 

63 
31 

no 



25 2 : 

J.....: 



yes 
2 
2 



yes 
yes 



37 

25 i 

^ ; 

yes 1 
randoni 

10 i 
12 I. 

0 : 

"'T- 

9 i 



25 

yes 
2 
2 



yes 
yes 



19 
10 

yes 

6 
13 
1 



25 



yes 



yes 
yes 



17 
2 

yes 

10 
12 
3 



10 



^ Table 

Six Approaches to Solvinj; tho (15,15) Sanple Problem 




Aooe'ndix? Oef inT tion fif laiUfi. Tgrmg <10>* 



. A . lQjtTca_l >' inequality in bivalent 
: variables is meant to be an inequality 
with 0; 1, -1 coefficients only. 

."A preferred ^ vaV iable inequality^ 
(abbreviated p.F.) is logical 
Inequality of the form ^^ytj) ^ 1 # 
y(j) being either y(j ) or 1 - y(j), 
which" expresses . the^condi tion that at 
least one of the y(j), J ^ J, must be* 
one* > . ^ 

. The* values 7( j ) ■ 1 (i.e., either 
.y(j or y(j )«1, depending on the 

nature of the 7(j)> are germed the 
suggested or Indicated valueso' of . the 
preferred variables. 



G'lven some procedure for ^eneratin^ a. 
set of preferred inequalities, we caVl 
minimal (relative to the procedure) 



tHose 



p. i . 's with minimal number of 



non-zero coefficients. 

The fiezree jd of a m.p.I. system^ (ahd by. 
extension the degree, of the initial 
ineqiial ity or system of 
from which the m.p.l, 
derived) Is the number 
coefficients of one of 
preferred Inequal Fties. 



inequal Ft les 
system was 
of non-zero 
the minimal 
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; ^. : ^ ■ . ' ' ABSTRACT ^ 

.In this paper .we 'study an accelerated 
version of Powell '-s con jugate .airection 
technique for solving unconstrained non- ' 
.linear problems. This technique employs a 
-method-^f taking large steps to enhance 
movement along ridges . Not only -"does this - 
, technique improve: the speed of convergence 
.for nonquadratic problemsv bxrt jLt improves 

the robustness of the procedure. An APL 
^ode^ useful. for research because of its 
. interactive capabilities, is presented and ■ 
described in detail. ■ \ : , 

: J... INTRODUCTION - ' ' . * . . . / 

. • , The topic o f non 1 ine4r un-cons.tr ained 
problem , solving is hy no means new to the 
OR/MS literature. ; Indeed, many of the" 
basic concepts- of solving this type of , 
fjroblem have been known- for some time.' 
Beginning approximately in 1960 there has' 
been a prolif eratdon of papers on uncon- 
strained optimization techniques. Of 
these, ^ one, of tiie, best Jcnbwn and most 
extensively tested, is Powell'.s method of 
conjugate directions. . This. paper studies 
Powell's technique and -offers a modifica- . 
tion that .qui: experiments show, improves - 
the rapidity of convergence to a solution - 
when the criterion function exhibits non- - 
linear ridges.: it employs a code that 
displays,>the- robustnes's of the technique 

♦Queen's University, Kingston^ Ontario / ' ^ • • 
.Working Paper No, 76-4/ May fgjS. \ ^' 



as well as effecting 6{ir particular means 
. .Of accelerating- convergence. The former 
;is. demonstrated by- a -discussion of the " "** 
effects on program operation ■ of." changing " " 
various computational parameters. ■. We- also 
/demonstrate .the usefulness of interactive 
langua-ges, APL'in particular for this. ^ 
. kind of research-.. .* . 

Poweil"'.s: (1965) con jugate direction 
method is chosjeri. for several reasons. 
Among other tests of. Powell's method, - 
i Box's :a966):,paper indicates . that it is 
competitive .with, several- other methods 
^on a, variety of nonlinear problems, 
-Additional discussions, of conjugate - 
. gradients can be found, in Fletcher and 
• Reeves (1964>. Powell . (1965:) and more 
recently Zangwill'- (1969) . Moreover, 
Powell's technique is straightforward and 
can' be implemented rather easily without ■ 
requiring either large.. amounts of core . ' 
storage or extensive manipulation of 
stored variables,. so. -that i-t constitutes " ' 
a reasonable choice for solving large: - . 
scale problems. Another -reason for the 
selection of v^>ow^ll 's technique* is that 
It -.avoids^ the necessity of. calculating 
.derivatives. Finally,, the conjugate " ' 
direction-technique ha,s^-the- property of 
finite ..convergence for gu'adratic " 
functions; in- this sense it is ' similar to 
other techniques, including those "by "' 
Fletcher and. Powell (19630 and- Fletcher • 
and Reeves -(1964)., although the" latter 



two cases make use .of explicit calcula- 
tions of the first derivative. 

The outline of the. paper is. as 
•follows* In section II we briefly dis- 
cuss Powell's' algorithm and the minor 
^ m9dif ications .we. made in setting. up our 
comparison standards 'In Section III we 
examine the ridge >fol lowing problem and 

. our method of accelerating the con.ver- 
gence of Powell's technique when 'these 
.ridges are. nonlinear . The results of 

; tests on several example problems are 
reported. In section IV we discuss some* 
problems caused by variation in certain 
input parajneters. Section. V presents our 
position regarding the usefulness of APL 
for carrying out the type of ' investiga- • 
tion reported here. Finally Section VI^ * 

. presents our conclusions.. The APL list- 
ing of the code that generated our 
results is included in an .Appendix-. 

II THE TECHNIQUE 

In this section we present only 
enough of Powell's method to be able to 
identify the differences between Powell's 
original code and the version we have 
employed. The major difference, leading 
to accelerated convergence in problems 
with nonlinear ridges^ is permitting 
PowelL's algorithm to commence a new ". 
iteration-at a point different from (and 
usually worse in terms of function value) 
the points at which the preivious iteration 
ended. This modification, which we call 
*a\leap,- is discussed fully in Section 
III* ^ In order to demonstrate the com- 
parability of our results, it is 
important to indicate the . additional , 
minor changes to Powell's technique that 
we.have foxmd useful. Essentially, these!' 
minor iriodifications are designed" tb-^^ 
" perm.it: making certain choices of .com- 
putational parameters that we employed, 
in .developing the accelerated conver- 
gence procedure. Powell's basic code 
may be. expressed in the following, steps. 

i) For r = 1 to n calculate' a minimum 
; . along direction- d^ beginning- at the 

. resulting minimum of the last- ' 
direction searched. The initial 
point is arbitrary while, the start- 
ing directions are the coordinate . 
directions of the , domain of the 
search • For any iteration, let p^ 

be the first point. and p the final 
point. ^ . ■ . - ^* 

iij If this is the first* iteration, re- 
compute the one dimensional minimiza- .. 
tions using *the initial set of * 
-.directions. Otherwise, calculate a . 
-new direction from (p^ - p^) and 

minimize the objective function in 
-this direction. The resulting point 
-is Pq.. for the next iteration. ; 



iii) Of the r + 1 directions select one 
to be removed by the criteria given 
■below and return to i) .- 

•Our modifications to Powell's method 
involve the manner in which directions 
cire removed, choices of some computational 
parameters, and convergence criteria. 

► ■ . ** . - 

Criteria for Removal of Directions . ' 
, Powell presents a scheme for determining 
which .direction .shduld be removed, a 
scheme .that includes the maintenaqice of ' 
the original directions. Our criteria 
merely removes (from the set of'^r original 
directions) that one which Is most nearly 
parallel to the new one (i.e., the 
smallest angle be tweeny the two normalized 
direction vectors) . 

' Tolerance Parameters . We use. the '* 
term - "tolerance parameters" t'^o refer* to ■ 
those parameters which the . Sperator; jaust * 

, choose before runnings the .program.' v,. ■ • 
Powell.' s : technique' employs well- defined*'' ' 
-rules^for modifying these parameters 

^ during the operation o:£ the- .code? . but the^ 
tolerance parameters theinselves are rarely 
specified in any discussion of the tests 
of proposed codes (in Powell's work or 
elsewhere). ■ We believe that in most 

. instaiices robustness of a particular 
technique is an important feature of its 
operation, since if" particular parameters 
must be determined before a given routine 
will work satisfactorily it can- imply a 
great deal pf testing on ' pairticular- 
fuhctions before a good selection becomes 
possible. In duplicating Powell's 
technique we experienced difficulty in 
determining the exact computational 

; parameters to ^employ, but partially- over- . 
came the difficulty through using the 
interactive properties of APL to obtain 
comparisons with Powell ' S: reported test 

.resialts on an iteration by iteration basis 

Five. different parameters were 
specified, for the operation of our pro- 
gram. In the listing of 'the program in the 
Appendix 'the 3 - vector TOL is used to 
specify the first threei of these 
parameters. These values are also used-, 
by Powell and are termed m, q and e in 
his paper. The last two parameters we 
employ, are unique to our code and are 
, called MPASS and MTE ST respectively All 
■five parameters are" usbd' during the- 
'operation o± the one-dimensional /search 
technique used- in our procedure.. A brief 
description of these parameters follows: 

TOL [L] is the upper bound on the 
length of a step taken by the one 
dimensional search technique. It is 
invoked whenever a quadratic prediction 
is further 4way from^ the points used- for" 
the prediction than the value 'tOL [1] 
permits. 7 • • 

TOL [2] is the magnitude of the step. 



^ ;iength along the direction being minimized, 
this vailue is used to determine the 
position of two additional points needed 

.for the first quadratic fit, 

TOI. [3] is the required accuracy to 
which veoriablas along this line must be 
determined before the search. for a ■minimum 
will be terminated (unless other termina- 
tion criteria are invoked). In the 
operation of our algorithm each of the 
above parameters affects the search 
dynamically in that they are employed in 
conjunction with the direction vector 

^currently used.; The end result is that 
tighter bounds ;are implicitly invoked as 

jjthe movements of the varicUbleis become ^ 

^smaller. 

MPASS is the maximum number of 
function evaluations allowed diiring ciny 
normal linear search. Our use of this 
parameter resulted from the discovery that 
.the nuniber of quadratic fits used to pre- 
dict a particular minimum along a line was 
a more useful parameter them TOL [3]. We; 
find that the niomber of iterations is much 
easier^, to contxol directly rather them - 
. with the use of , TOL [3]. Moreover, by 
. determining the number of gT;iadratic fits 
we use for any iteration, the accxaracy qf 
the search ccm be controlled in any event, 
as we shall show. 

> • ■ 

MTEST is used in an attempt to improve 
the robustness of the program. In the 
program me allow for ,a reducliion of TOL - 
^ 12] by* a factor of ten whenever TOL. [1] is 
invoked or when a predicted minimum tiirns 
out to be larger.. than any of the three ^ 
predicting points; '^The result is that ^ 
within a* single lijiear search TOL (2] l!^ 
scaled down if unreasonable predictions 
have been achieved. MTEST limits the 
number of times this reduction can occiir " * 
for any one linear search. 

^ • STOPPING ROLES . NO specific routine 
was written to stop our code, so that the 
result is a termination by default when-, 
ever no new minimum is generated in any 
conjugate direction. Stopping tinder 
these conditions is a. result of machine 
accuracy. It. would be an easy matter . 
Either to invoke stopping under less 
stringent criteria or 'to use Powell's 
-stopping rules. ' Because" of the nature 
of our research, however, we chose 
merely to input a specified number of 
iterations and include the capability of ' 
restarting the program without loss of 
previous information: This procedure 
might also prove practically useful diiring 
batch processing. Especially, the ability 
to stop the operation cmd perfoni some 
cost-benefit cmalysis before deciding * 
whether to continue the search might be 
useful when the nonlinear program is 
working as a subroutine to k more general 
program. * . ^ 



Duplication of Powell's Results . 
Table I displays the. results o£ using 
our version of Powell's code on the three 
. test problems reported in his paper / . The 
results of our -code use TOL - 10, 
.OQl; MPASS a 4 arid MTEST =1.. The 
simiiairity In number of evaluations fis ; ' 
of course, not siirprising, cmd. is included 
only to demonstrate the similarity in our 
procedures aLS.used' so far. In the rest of 
the paper compeurison standards 'will- be the 
results obtained using this' version of 
' Powell's code. -The'se results will be 
compared wi^ those obtained from using 
•the leapslze modification we have devel- 
oped to accelerate its convergence- ' The 
nature of this modification" is discussed 
next . ' ^ . 

Ill AN ACCELERATION . TECHNIQUE FOR THE 
CONJUGATE DIRECTIONS SEARCH 

A* well known phenomenon in nonlinear " 
search is often referred to" as ridge 
fol?.owing> Xn many problems the search 
begins to- follow a Ion? narrow'ridge (or 
valley in the case of minimization), with 
the result that the program runs for an 
inordinate length of time while improving 
the criter^ion values very slowly. The 
• possibility that chajiges occur so slowly 
that the program terminates far from an 
optimal (global or local) solution .is also, 
present in these cases. ..While conjugate 
directions work well in the case of linear 
or. near linear ridgesV' whea tbe ridge 
curves conjugate directions can tend to " 
zig-zag considerably/: jnuch as gradient 
directions perform in the case of quad- 
•ratio dictions." Figure I provides an. 
hypothetical example of a -typical se^ch 
pattern in this case. The modification of 
Powell's technique that* we report tierein! 
was predicated on the belief thit' it might 
be useful to continue to move in the 
general direction of the ciirvatiire of the 
ridge for some distcmce past the exact 
minimum along the search direction. We 
. conjectured that this procedure will lead 
to a result somewhat like. Figure lb, (and 
thus improve convergence) whenever the 
function is, not too closely quadratic. 
In l±Lese circumstances, the ridges become 
nonlinear and would, we expected, slow 
.down convergence of the unmodified con- 
jugate directions technique. These 
con jectiires were validated: Figxire 3^1 - 
. provides some actual results of a search 
i.we conducted on the Rosenbrock problem. 
The results of this cmd other searches 
are discussed below; supporting data for 
Figure II appear in Table II (-Leapsize =^ 
1 and 3) . . . • ^ 

In our modification to Powell's code 
we move in the general direction of the 
ridge's curvature at the end of each ' 
iteration. At this time'*ar new conjugate 
direction has been generated and we have 
minimized- in this direction, our 
modification proceeds by continuing along 
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TABllE !E: V Comparison of Two: Programs to 
Demonstrate- Similar .Results . ' 
'All problems taken from Powell < 
(1964). ' ' . ^ 



*Our^ number of evaluations is adjusted to reflect therfact". 
that 'wc made no use of the unit second derivative as Powell 
does. TJiis leads to one-less evaluation" along any direction 
previously used. 



this direction some number of units 
further than the distance we have just 
moved* For instance, if in minimizing: 
along the new direction we had moved a 
distance of one-ha^f unit, at. the end of 
the iteration we take what we term a leap 
to a new point in that direction (usually 
with a worse function value) • ' The length" 
of the leap is called a leapsize and 
indicates how many units we 'use as a 
factor to multiply the current direction 
vector to irfove betweefilPiterations-* The 
effect' of using this parameter is 'dynamic, 
in the" sense that . it depends on the size 
.of the direction vectors calculated in. a , , 
given iteration^ Thus, as the search ' 
gets closer to a minimum, th^ size* of 
leap actually taken reduces to zero* . 

Table II reports the results of our 
modification on Rosenbrock's (1960) two 
dimensional problem:' • * 



Minimize^ f (x^ ^x^ 



) = 100(x2-x^^)2+ (1-x^)^- 



In our progrcim, when the leapsize para- 
meter is set to 1/ the seaurch technique 
duplicates Powell 's* If leapsiz^ > 1, 
each new 'iteration begins at a point 
different from that choscfn by Powell's 
method.- k 



This",fxiriction is of interest because it. . 
has the type of nonlinear valley we have 
been discussing. In Tabl-e II we compare 
the results 'Of using 'Powell ' s. method with 
the' modifications resulting . from taking : 
progressively larger leapsizes. It, can be' 
seen that Powell's unmodified search: » * 
technique moves rather -slowly' along the 
nonlinear valley of ' the Rosenbrock. 
function while a(s expected, the use of the 
,deap moves the search more rapidly along • 
the .valley, thus in general effecting, a - 
more rapid, convergence' to a minimuia. . *, • 
f - ' . ' ■ ' ' ; • 

. Tables III and IV display the CLf feet 
of leapsizes for a three and a four 
vELtiable problem, respectively on which 
test results axe also, reported by Powell 
(1964) • The -three dimensional problem is: 



Maximize f = 



l+(x-y) 



+. sin fl/27rxyz} + 



exp' {-C 



x+z 



2r -v.- 



while the four dimensional problem is of 
the form:. " . 



; Minimum 



Star 




Start 




Minimum 



. (a) 



Cb) 



FIGURE I: Possible Improvement in Convergence 
^ Ca) Following the ridge; (b) Moving 
'away from the ridge. 




Powell^with 
mod if i cat ion 



FIGURE Ii;: , Convergence of Powell ^s Technique* and 

, " its ^Modification to Solution of Rosenbrock . 
Test Problem. Path chosen by Powell 
• * -'■ indicated by dots; Modification by crosses. 

^ . Dashed line indicates continuation of movement 
' ^ made by the modification. 
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" ; , ^w?n) ^ ' LEAPSI2E .:2*' - - •. , • leapsize-o* . . 

..NUMBER OF ^ NUMBER OF ■ • ' ' 'NUMBER OF • • 

EBAIIOS FUSniO:i.: , . FUNCTION FUNCTION ' .'■ FUNCTION FUNCTION . FDNCIICN 

...EVALUATIONS x. x VALUE". EVALCATICNS r %• VALUE- .EVALUATIONS x x VALUE 

^ ■ -1-200 .1,000 '24.200^ 1, ' -1.200 .1.000 24.200 , 1 -1.200 1.000 24.200 

i ^ . ■ ' • ' i 

23:.. .>-.837 .677 3.435 ./ 23 .^^ .837 .677 ' 3.435 23 .- .837 /.677 , 3.435- 

-„ •• , A ' • :• V. ^<-..68't ^.372 3.750 ) ( -:S3r. 067 . 6.922) 

. --66^ -56^ .3.019 . - 3,6 . -.495 .206 2,387 36 -.239 .037' 1.576 ' 

_ •'*3'+ ".122 2.m) . (,..199 .-.006 1.6«y 

-, . ,.<49 -.372. .098 ^2.040 49 ,177' - .003 1.508 . x49 ' .124 -. 008 .822 

••: - (- . 082 -..066 1.703 ) (. 223 -. 020 r.C96) 

51 -.090 -.027 1.310 . 62 . 080 - .028 .967 62 ■ ' . 319 .077 .526 

. (.166 -. 037 1.107 ) ' (.560- .182 1.941)-. 

73 .168 - .002 .. .783 75 .308 ,068 .553 75 .572 . 308 -.218 

■ ' f -"^^S .138 1. 038 ) ( . 697 .it23 .1^85) 

85, .352 .088 .546. ,88 .525 .254 .272 88 .759 .564 ;07S 

" : ■ ' ■ ^-613 . 329 . 352 ) • ■ ( .907, .765 '.345). 

97 -^62 .222 .298 . 101 ■ .716 .496 '.110 - 101 .929 . .857 .008 ' 

t . 819 . 52-6 .235 ) ( l.OOO .980 .039 ) 

106 .608 ..382 ,168 114 890 :782 . .020 114 1.000 l.-QOO 1x10-5 

J • . f . 967 . 909'.. . 052 ) ^ 

in-^' - *.660 .427; .122 127 ..995' '.989 2x10-^. ■ 

., ' .V (1.028 1.-054 .■ '.002 ) 

.* 131 ^ .783 - .599 . .067 ■ 140 1.002. 1.003 5x10-6 ' 

(.993 . 983 itxlO-'*-^ 

■143 ,895 .802 .011 153 1.000 IVOOO 1x10" 7- ' ., . • 

155 ■ .959 . ;916 ' .003 



167 ; .?96 .991- 2x10-5 •.. • " . 

179 .999 .998 3x10-6 

191 . , 1.000 .'1.000 2x10-11 . •. ■• 

. ■ , ■:. • • . ■ ■ ■■ . ■ ■! 

Table II: Soliit^?.'; : ' Rosenbrock's Function with various Leapsizes 
_ TOL = 10, .1, .0001, MPASS = 4, MTEST = 1 ' i 

, ' *Points in pare-th-ses indicate the beginning of the next iteration. 



LEAPS IZE 



LEAPSIZE 



NUMBER , or A 

FUNC^-ION 

EVALUATIOMS" 

Up 

59 ^^ 

. 73 
88 

120 



FUNCTION 
' V^ALUE ' 

2 .'859 

sea ' ■ 

2.965 
2^983 * " 
3.0CO. 



NUMBER OF ■ 

FUNCTION 

EVALUATIONS 

■ 41 

58 ., ^/ 

* 90 . 
109 
125 

-lUl- " 



■FUNCTION 
VALUE 

2.639 

. 2.6U0 ^ 

2.581 

.2:715^ ^ 

2^.9 75 

Z.9'99 

3 .000 



NUMBER OF • 

FUNCTION 

EVALUATIOJ^ 

39 : 

• 55 

. 7^ 

. "89 

109 

: '128 

143 

^ -158 



TAL3E ^III: • Solution ,cf Three ■ V^iri able P'roblen with 
various Leapsize^ TOL = 10, .1, 1 0001 
' . MPASS = U 'V 
... MTrST. i.'l 



LEAPSIZE t ^•:i:- 



.'LXAPSIZE 



NUMBER ■ OF - 
- * FUNCTION 
ITERATION EVALUATIONS 



■ ''NUMBER OF.' . ' 
FUNCTION FUNCTION.' FUNCTION* 
VALUE' \" EVALUATIONS VALUE 



FUNCTION 
VALUE 

* i:93U ' 

2-^629 

2.728 

2.729 . 

2*325 

2\ 995 

2.999 

3.000 



2 

6 
8 
10 



91 
129 
168 
203 



. c ■ 

-.^0075" -"V 

.0018 
s-5 



UxlO 



2x10 



-5 



■ S§ 
92 : 
13'3 
172 

209 



vlUOl 

:^.oo2i , 

*0018 
,-5 



.6x10 



5x10 



«5 



-LEAPSIZE = U 



ITERATION 



2 

5 
8 



NUMBER OF 
"FUNCTION 
•EVALUATIONS 



■ , ■ 55 

' ■ / . 95 
135 
17U 

TABLE- IV: 



FUNCTION 
VALUE 



..lost; 
'.0 012 

";qoo3 

.-6 



LEAPSIZE 

NUMBER OF 
. FUNCTION 
EVALUATIONS 



55 
'97 



FUNCTION 
VALUE 



.11111 
9806. 



UxlO 

Solutrion^of Four .-Variable Problem 

with Various LeaDS-izes 

TOL^ = lb", .1, -0001 ■ ' , 

MPASS '= 4 — " 

htest'' = 1 



\ 



■ ' .^"^ ■ ••• ■•. '-^ ■ ■ 
+ . = ' Solution near lihe^ problem 

• -= Sl:ai;ting:. point . • 

il* = Soiution nonlinear probliaiti ]^ 
■ ' ' ■ ■ • • - . - 
• = stating" poi-nt : - 




■FIGURE III: SIN X +' COS X 



ITERATION 
i 

■ 2. " -. 

3 '-: ~ _ 

5 :'■ 

■ "7 

8. 
' 9 
10 



NUMBER OF' 

FUNCTION . 
■ EVALUATIO>IS 

- ifS • 

S9 

93 

•117 

' -IHl •■ 

164 

- 186 - > . 

■■■ 205 ■* 

i 

-■a 

244 ■• 



FUNCTION 
VALUE 

1S3..384 • 
82.772 ^ 
^32V300 

- 15: 084 . 

■'.285- 
.005 
■ .003' 



3x10- 



-3" 



3x10 



-9 
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T/3L:C V: Sol'ution of Random . Trigonometric- 
Using Powell's Method . 

^' . " • . TOL = iQ,. .i; .oo'Oi ■ ■'. ■ 

MPASS =4 
■ MTEST-= i ■' _J>.v 



Problem 



: + + lOCxj^-x^)"^ 

•. . Inspection- of . the tables shows that the * 
. s^e. leagsize .-is not necessarily the best 
. for every problem, m Table TV: a leap- ' 
.;si2e^of eight moves the point so far from 

• "the ridge that it . canxjpt^ be attained 
again.; We. would pbviously wi'sh to lower" 

• leapsize whenever the final point in an ' 
Iteration (before the .leap) is worse than 
that of the previous iteration. This will, 
.of course^ assure convergence if a local 
minimum, is also a global minimum.-. 

-Heturriihg now; to the three dimen- 
sional results reported in Table lii, we ' ^ 
i|ee that leaps do not improve the rate of' 
convergence in this case. We interpret' 
this resultr^s stemming" from the fact . ■ 
. that this particular function approxi- 
mates a quadratic in .the region we are - ' ' 
searching.. . Supporting this belief is the 

- fact that the first complete set of con- 
jugate directions is generated at 

- Iteration five, whereupon the search 
converges immediately to the optimum. 

To justify our interpjTetation, we- 
show the case of quadratic convej-gence 

.mdre dramaticaUy for a function which is 
not obviously quadratic. Ta^effect this 

.. demonstration we use the bounded 
trigonometric j|£nnction presented by - 
Fletcher and^-jSlRrell (1963) . m this case ' 

■ the function is: ^ ^ . . 



n 



Minimize f = 



.n 



i=i j=X 



(A 



sm 



X. .+ 
3 



-cos x^) 



where ' the A and B matrices are random^ ^'^ 
numbers between zero and/100 and the 
solution's 



-TT and 17 r 



x^j are random ^variables between 
The values of. E ^^tare then 
. determined so that each' term- will equal 
zero At the optimum.,: The starting, point 
for each variable .-is^.ran^dbm^ized bet" " 
+ .It: and .Itt 'of ;the;>bgtimum S0I5I 




J^' Although, the 'pfoblemV is' s^ 
. 'honquadratic/ 'it .turns- out to be' 
difficult,to;genei:ateVa. random problem • 
which IS not. ^ essentially quadratic. 
' . K^^^f. Iir shows the . fu;iction sin x . + • ■ " 

. cos'Xj and the Values of' for what ^^^^^ ' 

- found as a y^ypically.. genera ted random' 5> ' 
. . dwneneional., problem., '{Results for this. '. 
; ./probl^ a;:e reported- in Table. V^ ) Note 
• .1=^.^t all these values fall, iii* th^\ iiiiear 

• Mrtion of the curve. - Although the values 
PRir\^2^-^^^^ ^^^^ effect pn the shape 
Cf^ bh;LS curve thg^f aqt that there are five 
^ . :^"w*V terms^_i5: ;eeeTi' 'squared tera ,us , 



to believe that the nonlinear ef fects.^can 
be balanced against each* .other. * The 
resulting function is then approximately 
the square of five linear' terms - a , " ■ 
quadratic. We would expect that in a. 
conjugate ^search; procedure the optimum \ 
would be found as soon as a complete set.' 
• of ' conjugate "^irectic^s were fbtmd. As can 
^. be seen, in Table V this 'is exactly the- 
case, - At:, the -nd of the sixth iteration a 
complete set of five conjugate vcfirections 
. has been fountT^nd used - at this, point " 
the procedure moves- nearly instantaneously 
^ to the optimum; No leaps are displayed 
for, this function since -they, are> of 
course, not going to' improve matters any. ' 
We might conclude from this experience 
'that perhaps leaps should be ignored until 
after the first set of conjugate direc- 
tions has been developed, -The study of 
this problem is interesting particularly 
in lig^ - of its considerable use as a test 
functix.i for nonlineair problems.^ Ixhe 
results of these tests will tend to^e " 
: biased in favor of^ quadratical^y^cori- 
. verging pfethods. , " . ^ ■ ' 

■ ■ .For our final test; in this series we. 

' again used the trigonometric function just- 
discussed but this' time choose . parameter s* 
so . that the solution will be on the non- 
linear portion of the curve as shown .in. 
Pigurelll. Now the no leap method for 
this problem took 435 evaluations to 
achieve a locally optimal solution of 
• 6706, as compared to 226 evaluations to *' 
achieve the global optimum- in the near- ' " 
linear case. The use of leaps yields 
dramatic .improvementf;^^^ in the, operation of 

■'"the code/.. Table VI displays, the results - 
of this investigation.^* Powell ' report?- - 
solutions to two five dimensional .proi)lems -. 
with, as few as 106' -evaluations - which 
-accordingVto our estimates coizld only* be 

r achieved . if ^ the function were -perfectly 
quadratic in --the region of search.- This' 

. '^resuXt emphasizes onceMiiore. that iTn noi^- • 
.quadra1:ic cases leaps ^are useful while ' 
in'.quadratxc cases where the ridges are * 

■ linear they tend to^be of little or ' '"-.^ 

. slightly negative, value*- " • " 

J . \ the next section we. explore the 
effect of our leapsize procedure on some ' 
more difficult exponential problems, for > ' 
which the iinmodified conjugate ^directions' . 
technique^ has faired to locate an optimum. 

' IV 'EFFECT OF TOLERANCES . ON - ^ 

; SEARCii PROCEDURE . . . . i ' • : 

' No technique is impervious to the- 
. selection of program parameters. Not 'only' 
can tolerance parameters af feet ' the 1-ength^' 
-of time taken, to converge, but they; also 
can determine whether , a. particular ■ ' 
technique will cofiver^ge. at all..' rh^Tstble' 
11 given earlier we presen^ted the results" ^ 
(5f our program" on the-\Ro^enbrock ftiribtion. • 
Tolerance characteristics .are iairlv 
robust for this problCT. Leaps, in tKis 
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(POWELL) 
■NUMBER OF 
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NUMBER "OF 
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• FUNCTION 


FUNCTION 


' 'function. " 


function ■ 




ITERATIOri " ~ 


- EVALUATIONS 
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table VI - Sqlutiiori For -Contrived, Trigpncimetric •Function .-^^ 

yj.-" ''"•'■'" -'^ ' . '.^^ '• ' 'tOL = 10» .l,"'.0O01- 

^Stopp.ed when no variable "changed more than . 0001 ' ' MPASS = 4 

■'-■-..,'■. .^.■.■■--' ... ^ ■ • . „ ■■ . ■ . ■ .J4TEST =■ 1 



case also seem to be robust in' thaf^they 
■ improve performcince in all 'instances^ 

To study the importance of toler- 
ances in more complex circumstances we 
. used a probl^ generated by Box (1966)^ 
In-Box's notation this problem is 

Minimize f (a- ^ 



1^ ^3^ " 




-X -lOx. , 2 
-e )] 



, ^ " where the stimmat ion 'on , is . from 0.1 to 
/'^^^ 1^0 in incremeiit^ of ...b.l.\j?his problem 
.. r has . inf initely -many*] global, and 
" i . ' bptima'-, - We , chqpse thi s problem ' s ince Box 
^ I^Y^^ipor ted;. that f^^ six of nine starting 
fc I\l v> tints ; Powell method 'failed* to converge 
■k mmmm^m^ . Qf^. the global opt?4na . ' Table ^Vll 
'a!nd VIII shows the results for one .of 



these cases, with various parajneters.. 
As -did Box, -we had difficulty .getting 
solutions for any- of *tfies.e . problems^' 
when no leap was used/ altiiough;^we did 
find that 'certain 'tolerance combinations 
^achieved solutions .better . than- that 
'reported by Box- We also fptind that in . 
many cases scrapping the' conjugate 
directions from time to time and re- 
stating the search using coordinatqi; 
.'directions seemed to improve the rate of 
convergence. This' obviously implies; that 
coordinate directions are useful for this 
problem. , Use of leaps improved conr- 
yerge:nc'e * in all cases. . In some cases 
'convergence ' to one of the global op;tima 
occurred through the use of leaps. ^In. « 
almost .all cases using a leap 'led -to an. 
.improved value of -the objective fuiis:'tipnw 
■ To improve things further still; we^, - 
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NUMBER OF 
V .. .rUKCTION. 



I^APSIZE,, • EVALU.'^TIONS; 
(Powell) 251 



/value. .OE ■, . 
FINAL SOLUTION ■ 



116 

-.179 

■.178 
'l53" 

.187. 
162, 

313 

. i9i'. 



- 3x10 
9x10 



-8 



-9. 



.3x10"^ 



5x10 



7x10 



-6 



3x10 



-r 



2.X1.0 



1x10 
8x10 



-2 



-7 




4x10'®- 



Fomd- no" -min the ^Lt iteration 

'"■ing. this large a step. ■ / . ' ". ■ \ 



• Upper- bound dn ■ . ' TA'BLE VIl-: 
, step size 

Step length-'for / . ' 

£irst' quadra'tlc\fif - , ■ „ 

in direction " i_ •• 
■ ' ■ ^ ■ <s 

'A'ccuracy of ■ iiriear ■ ^ 

. search " ' ■ ' ' - " ■ .: ■ 



■.Results . From Three' -Dimension^ 
Exponential MTEST .= -l"--'- " 



]( Box :-19 66; ' Start 



. . changing leaps i;^would be advisable., ' Tf - 
;. a leap produced '^better solution along' ; ■ : 
■ J ; some direction, we should continue ■ to leap 
.Tintil.a worse sol.:tion is found before ' ■ 
continuing on to tae next iteration. When^ 
* ever we performed this operation '>re " 
' ' -achieved convergence with Powell's code. 

.' ^ The state of affairs just reported . : 
points out a problem with the one- ^ 
. j dimenisional qxiadratic^f it.used both by. 
- • Powell and by "ourselves. The surface 
f iseems/ to \te s flat along conjugate 
" directions'' that; the i?^ear search stops 
, f.ar. from, the- optimum along -tfie -^Line. Thus 
V.i-t is difficult to say whether conjugate 
. 'directions with a different l£ne sesirch . 
V,;' routine would not work with this type of 
•/. problem. At emy rate it is encouraging 
to^ note that use of leaps not only 
r . improves speed, of convergence but has the 
. effect of creating a more robust search 
O :^ure« ^ . 
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; y • USEFULNESS OF INTERACTIVE COMPUTING-' • 

IN- THE EXPERIMENTS ■ V ' 

•:; / The line of. research presented in this*/ 
paper would have been nearly . impossible 

/'to conduct without using' a highly flexible/' 
-interactive code such-as APL;- In many 
ways the study of optimization is typical 
of, highly complex search, evaluation and 
decision making tasks ^ where, a great deal 
of information both useful and spuriouis is ' 
available at some'cost. Gathering and 
presenting this information to the 
decision maker may be at . least ats .i 
difficult as understanding the theoretical." 
and analytical aspects of the problem. . 
The interactive facilities of APL helped 
in^ ntimerous ways in organizing these tasks. 
Firsts we found it easy to develop sub- 
routines whose. performance we could study 
in detail prior to including them in our 
main search routine. Both debugging and 
operation of these routines could be ^ 
carried out on an almost instantaneous 
basis because of the language-",s inte:;;-: /. 
active, capabilities^ and its. abilitv' to . , ' 
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.10' 

io. 
: 10 
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■ .•: 3 

■io- 

•10. 



■'■ '■ ■'TOL '■ ■ 
2 3 * ' 

.1 -.0001 
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.55 .0001 
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1 (Powell) 
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^> Upper l)oi2nd on 
. ■' step/size " . 

2 =/Step length- for 

^ ■ ' first -quadriktic. fit 
in any . direction ; 

3 = Accuracy of : linear 
. -search- ' 



table' VIII:. Re'suits From Three .Dimensional (Box, 1966; Start* 2) 
Exponential' MTEST v= l ^ : 



subrd 



^ iiX 'detail t- z f ;;^ctioning o^. these 
5.uj-.iWL^£ines.^ ' Second,;. as , through exper-- . 
ience we learned which ^numerical ouj;puts- 
from, a given . routine wfere , important, to 
'monitor, the;^editihg 'capabilities of APL 
. flowed us rapidly, to : suppress ^ ■ 

- • inessential information ThirdS when ^ 

- an hypothesis regarding the oj>er,ation 'of 
r some- part of the search routine was-{ ^ 

* formulated, the language allowed- usr . 
instantaneoTisly* to test that-^ hypothesis. 

■ under controlled exper iment";al conditions 
in wfiich possibly compounding variables 
could be held constant, .^Fourth, the 

: ease with which subroutines could be 
, programmed arid, operated independently 
''allowed. us, in effect to develop and- 
utiliz^ tools for conducting ,these 
experiments,- /±us. helping us to 

■ ascertain certain problem characteristics • 
as :bur study proceeded: 'Fiheaiy, when 

. hypotheses wei^e shown .by controlled^ - 
experiments to :be' incorrect, the. editing ^ 

• afid memory ^ capabilities of • the ""language . 
made \lt easy to backtrack and hence 

' reclwer successful approaches before 
• vendue tijig ^w expe^ 



VI -CONCLUSIONS - . ^V. 

" " ' - " -"V- '''' ' > ^■ 

■ In this paper we > considered the per-- 
. formance of Powell's^ conjugate direction 
0 jjthod, as originally/ developed and as 
Jdified in the manner . indicated. Very ^ 
^A'W^^ problems could 'be ^found which gave 
fv'^Powell's original' tec^^ ;difficulty in 
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finally" converging to ab solution. ■.',Ih " . / 
those cases, restarting the directions or _ 
some other .'interactive parameter change 
would always start the search moving 
again - in this sense Powell' s technique . " 
never failed to converger However when 
we introduced th^ -concept\of leapsizes, 
the modification demonstrated a. beneficial n 
effect on speed of .convergence and the V 
robustness of coiivergence for the . ' ; 
technique :in. particular 'for cases where 
the functions exhibit nonlinear valleys * 

If jshould be' said tliat the concept of 
leapsizes is not a new one. For instance,' 
^ Rosenbrock- (1960) used, this* idea. His. 
usage of .leaps,- however J. differed from 
ouiTS in two ways. First he used leaps 
with a steepest descent. method. Second, 
he leaped only in an attempt to improve ' 
the value of the objective function - not- 
to move away from" thie ridge itself as, we 
do. This is a very important distinction.- 
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• APPENDIX: LISTING OF CODE > 

' • THIS P.^.EMPLO7n\?0WKT:LK': •Cnj:auaATF ,COOPI>TN^^^^ VARIABLE LEAPniZEi ' ' 
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SELECTION AND EVALUATION 



Chairma'n: ^: . A.C. -WiXliaras, Mobil Oil Corp* 

Panelists:- J.- GregoxY^ Cola-han, .Ge:r>feral;:Bat'tery tprp-V 
.'. >C-.R, Ellison, Mobil Oil C^p. ■ V, / 



Milton Gutterman, std. of Indiana ' 



i:>avid. Hirschfeld, Management Science Systenife 

Suinmary of panel discussion by A:C. Williams. ' 
• . ft. . 



: How does an organization .select a 
mathematical programming software system 

SLo^^T^^f^''''''^ ^ commercial environ- 
ment?^ What are-, the relevant factors to be 
considered, how can .they be^ evaluated, and 
.how should they be weighted in arriving at 
.; a conclusion. These are the questions 'ad- 
. dressed by the panel..-, On almost all mat- 
^^^^.1, th^re seemed to be al consensus of 
opinion,, even though ^he panel members 
came to the questipns from a' number of 
dif^ere^it perspectives. This note is an 
. att^em^it to' summarize that consensus,. 

As in so many ojiher ►din stances of 
work-a-^ay system -^valuations, there are-', 
three, aspects to be considered": (i) the- 
. ..functipnal, -(ii) the technical, and' (■iii)\" 
the- economics. These are listed-xn the"' 
■ ap.proximate order of importance. " " ' 

• . .. y 

Functional concerns, of course, begin 
asking, whether the particular system 
p^-consideration will meet .job reguire- 
fr both how and in the foreseeable * 
:e. This requires an evaluation not ' 
.^only of the system being offered, but of 
^ Its future possible path, of evolvement. 

• And .to. evaluate that requires that some 
rr^f, :qu-estion;S be aske6. aixJut.the^endo^. 

• wi;L1 , he rema^^i.': in -biifeiness? ' If ^he does ' ' 
'-wili;^ he -maintain- rt and- keep it up to date 
and. IS he likely to have it evolve in a 
direction suitable to our needs?' 

a j^/'^lf^rtant 't^. realize-' t^ . 
<CDir^ — "^^tiematical programming 
ttVly m IS -likely to influence and perhaps. - • 
HSHWa^fiuence gredtly, the future deveiop^ - " 




ment of applications' .in the-ccmpany. If. " ' 
a code has a good GOB f,^ature, large and 
comprehensive distribution models can be 
developed' and used.: A" general purpose 
MIP feature .could, aaiow development in ' 
directions- which would otherwise not be 
■.. possible^; The same could be '"said of a.'- 
quadratic programming option, . network flow 
subroutines, special purpose .MIP' s, a^ " 
separable programming option, etc. ' 

. An important functional qu'estion^ to 
be asked of a proposed system is how well 
■• -^tfcs^yj^.. integrated into the company's 
, . operating. procedures. • Especially in in- 
stances where results are. needed. on a 
".real- time; basis, 'as^ in some,.day-to da-y ' 
' ^®^^^2ry operations'/ we' need. satisfactory 
'. answers to these questions:." - 
. .(i). -Does the system facilitate getting ' 
data and maintaining it? 
(ii). Will it be easy to " ask- questions ' 
and get answer's (are the"m^tri?c generators 
. and the reptort writers satisfactory?). 
. (iifiif^.- Will the system be reliable? 

fa*. ■ •;^^^e..qpestion of . r^lfiabilitv gets trs' • 

mtoj^he technical questioos. * Reliability ■ 
• rs the question, of hg^K^-bug-'-free -i^ the 
code7 and hc% stable. is the dbdrT (sta- 
bility here refers to the codecs ability 
to handre numerical difficulties arising* 
from almb St.. singular.it ies\^d from degerr.-^- 
eracies). It is generally cpnceded that 
no coc^ IS bug-free' or stable for all 
probl»s, and. that^ therefore applications- 

^systems vhave -to be designed to take' those 
facts into account . A preliminary reading 

: ,on the reliability of a^systort can often 



• be obtaining be asking around- -. The 
•vendor ..will of ten'..sui>ply' a list of .satis- 
fied customers. However, it sHould be^ 

" borne in mind that .his liist of all cus't- 
. bmers and. his list of satisfied customers 
may not be identical . 

Many of'the "^old hands", keep" a num- 
•ber of test problems/ carefully safe- 
guarded f ran -any changes, to be brought' 
forth to ex ere i^se a proposed code. 
' Usually; there will be at least one test 

• problem that will upset the stability of " ' 
^ny code - the point being to see how " 
^ng it takes to bring it down, 'and how 
■yracefully it does down when it finally *• 
•.surrenders'.- . -\ . ^ • . ' : 

The test problems should, qf\course,' 
contain representative oroblems,- ^ypical 
of ' those tor be solved ir. ".producti5n- -Most , 
users- are not\su'rpr ised -if their- repertoire 

* of /problems turn up sane bugs in a code ^ 
bejing tej^ted by than for the first; time. : 
The . real questions are.: ' how doe's the • 
vendor respond. Does' he giv^ you a\ wqrk- - ■ 
around? Do "new bugs' Continue^l/to appear, 

..orVdoes theSfirst spate of bugs appear to. 
be all of- them? ^, ■•"-l'^ 

\ , Bug^ and instabilities will persist 
in every code, and they will have to be 

■ dealt with. An ^ impor'tant technical prop- 
erty, to be looked for therefore, is flex- 
ibility ta adjust torferances/ conveniently 
and easily; and to control the flow of the 
solution, by e.g., adjusting inversion 
frequency, level of multiple pricdng and 
other, parameters. 

While the local l.p. technical person 

■ wants, and needs such controls, in many 
instances he wants, to be able to s^hield . 
the users from- "worrying about^them. 
(Some w6uj.d say the- user shouldn't even ^ 
know about them," but this appears to be .- ' 
an extreme. view. ) Therefore, the ease 
with -which the system control language ^ 

* allows macros to be written and '^executed 
is an important feature -in most installa- 
tions. ■ . ■ <■ ' ■ - ^ 

In many installations the. ability to 
do recursion would be: an important prop- 
erty to be looked., for- in a code. In this 
case> it would be' important to be able to 
'read solutions, Jsp read- the shadow'^prices, 
to' analyse them^' a^nd^^to revise the ^coeffi- • " 
-cients conveniehtl'y and easily in a higher 
level language, such "as Fortran or - PL/1-. 

The economics^ is, thj^ matter, pf cost 
effectiveness. Benchmark runs have to 'be*., 
designed , run, and analysed . ' . Some -df\^h^. ' 
benchmark test 'problems will be for ^the \ 
purpose of exer^i^ing the coder i-e'./ 
testing it for bug is and' f of , its..stajDility/ ■ 
In testing for cost effectiveness, however, 
QT^ual productiori\conditioo^' should be 
"iilated as nearly" as possible.." Actual 

duction models should bejised^ i;E 

available. If most production, runs are 



made from an cdvahced steirting basis> 
with a few revised. elements, then' the' code 
should be tested under those." conditions* 
. The performance of the code in solving a 
problem starting from scratch is ^Imost ' 
irrelevant. If most production funs are 
made daring , prime shift in. a mitlti-' 
programmed environment, then the test .of 
■ the code should be done during prime .' . " ** 
' shift' in a multipro^rammed environment.-. :: 
Of course, results will vary from run^to- 
run, so--^that ^.several runs for each ^case ' * " 
* will .generally have to be made to get , an 
average and a- variance. 

/ , .Computation a^^cia ted with" the • . • ' 
mathematical programming. activities can • 

, : generally Ji^e - broken' down -in tq^ U) matrix . 
generation, '{ii)^ ^' execution ,*"'-'arici (iii) 
.report -.writingv -The "execution" can . ^ 

, further .be* broken down into "primal" and * 
"other" (typically converting the input ■ 

, . - data, into* the f ormats to ■ be used -by the 
-code,; .e.g-, SET UP, CONVERT,) 

■■ ; \ For at, least one, product ion shop it. 
was xeporte^. that the execution, step 

;,. accounts .for only aSbut 60% of the total " • 
CPa time charged to linear- prpg ramming,' . 
"the -mother 4 0% being taken up by matrix 
generation and ^report " wr.iting" ''And- only ^ 

. a total ^of - 40% was >pent in primal, i.e.,' ' 

. the aCctual execution of the. simplex 
method calculations. There seemed to be 
some 'agreement that these* kind of pro- J5 
: portions were not .atypical. The con- 
•elusion is, of course, that the matrix 
generator ^and report writers cost '"per- , 
formance can be significant and important, 
and should* be included in the benchmark 
tests, ly X 

What, tordo about tuning is a major 
headache in benchmarking codes* The . - 
amoiint of.main memory assigned can have 
a major .e|£-ect on the performance' -of the 
code,.andv of CQurse, an- inverse effect . 
on the'. other, work-' running in the computer. 
It is generally recommended that a nuijiber 
of different memory, size a ssignment-s be 
tested for the major application types, 
and that, on the basis of that data^ a 
recommended size be arxived at by -use of 
.the 'charge out system and/or ^negotiation / 
with - the "compu-ter center.." ; 

■ : - ■ ■ ^ '■ ■ . ' [ ; ■ ■. ':'■■:'' 

.V .4- How .much tuning .. of the parameters of 
the, code Itself - should be done- is a dif fi- 
cult question. ^ Which parameters are con- 
,sidered?most important are,; of course/ ^ 
; initially best determined by consultation ... 
.with ;the vendor or with other "uspirs/- -If . 
: the; range.; of* dif f erent probrem types, to • 
• be., solved ♦ is f airiy narrow> it can pay to. 
-spend considerable time and energy to' get . 
good*' parameter settings, whereas; if "the ^ 
pro*blem set,i.s. very variable^ and subject ' ■ 
to. 6hange,- it may T-be that one should, not > 
goj much ^l^eyond , using ' 1i:he-;^ef au It settings - 
■suggested byj-^he .vendor. ^ . ' - ;, . 

■ Overall^ tJT^g/bencM^2fk>^rAblem i§ ■ ' \ " 



difficult because of the many variables 
• involve^ and the fact that run results . r 
■ are thenlfely^s.. variTabie . Unless ■ great 
. care is tak^ii. in designing- the^ te^ts, the 
number'of runs ^nd the consequent amount ■ : 
of computer resource used can get out of 
. . An^? unless great care is taken in 

.interpreting and analysing ;the results, 
..wrong . conclusions can pe reacAied . . - , 

■ . xn suijmary, any organization with a 

■ .significant; present or contemplated -mat he - 

• .matical programming work-load will- find- it ' 
worthwhile to' exercise a good deal of^ care 
in^the selection of • a ' software system; 
-I^^issuefis discussed 'iji this .'note are 
/'•probably the. issues .that, the" organization 
Jteeds resolved-^.-* Buf, more important than " ■. 
■ .a^Y^:specific issue is; to ' assign th^ eval- 
uation task a" re^t'sona^ble priority and- ■ 
' budget, and to assign a good person to the 
-task- 



Attached, are' tables which. may be used 
■ as general guidelines 'to' some -of the ' 
mathematical prograram,ing systems and " ■ 
matrix* generators/ report writers. The 
tables are based on the best information 
th^t wa:s available at the time- they wer^ 
prepared: ' {Tew systems ^d new features ■ 
';are continually being brought' out.; however, 
sp. that ^any> potential user woiJld have to' 
obtain the most up to 'date. information/ " 
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ABSTRACT , • '\- . ' ./' 

There has been increasing interest in software developoient-techniques over the past f ew r 
. years. Effective techniques for developing reliable! maintainable and extendable 

software Significantly reduce total software xosts- Several 'techniques have been 
^scribed in recent literature- The objective of this paper is- to report experience with 
i|o»e of^ these techniques in the deNfelopm'ent of APEX-IIIi a large scale linea? prograraming 
.^.systen-- .• ^ ' . ^ ■ . • ' . 

• Several '^f the tecjiniques are felt to contribute to a succes^sfu^l development. They • 
include a project staff organization, similar, to a "chief prograranier team^i documentation 
and. coding standardsi and open programming. 

-;Te<:hniciues whi<:h ^did'not appear directly applicable to the development were structured* 
progr'aioing^ andCtop'down programming- \ ■ . . - 

■ ."-i -. ■ , . " - ■ _ < ' ' * ' • ■ 

FinalJvyV'dt^^is noted that a significant amoun of effort is required to prepare accurate 
estimialtes of 'development costs. 



IHTROJUCTION ^ - . - 

-Last'- year -r our ^dbmpany' completed develop- 
^•■nent' of .a largie scale linear programming" 
syilTSiilV-APEX-III- Several recently V 
publicized/mathematical programming and 
software -development techniques were . con- , ^ 
sidered and used in the development-/ Over, 
tjie past yeari -we have had the^ opportunity 
to'*evaluate the effectiveness of . th6se 
techniques. ^ ' v 



.A development of this kind requires two 
distinct types of sKiltsV m£(thematical , 
|Sj. prpgraifioing'^skills and' software develop- 
ment skills'- By roat,hematicar programming^ 

' skills*! we mean those skills unique to^'a 
linear programming sysiem. they include 
a knowledge of tlie t^.pe, of problems to be . 
^ solved* 1 An ef ficient^ design considers. 
/• model -characteristiQs^such as size-r • 

-density and structure. Familiarity with 

. characteristics of curreht algorithms is 
needed to^ properly match tbe •aLgorithm's 
to the. computer. Performance and 
numerical yfetability of a- Code reflect the 

" programmer ',s understanding of the algor- 
ithm- ' ' 

Just as iraport^ant^as the matheAatical, ^ 
programming skills-i arc the software - 
' ^ - ^evdopraent, skills* These are the . skills 



^ required for development of any large ' 
' prograraming system- They include a know-' 
ledge of the computer! the operating!^ 
system^! the compilers and assemblers- ^' 
They include ^familiarity with data 
handling techniques and means for 
' ef f rfciin$ use of the hardware- 'These 
skills also include a working' knowledge 
of techniques for, producing reliable and 
. maintainable software products- 

> Jhe purpose of this paper^s to report- 

our €)^perience with these software". 
. -development tiBChniques- ^ ^ 

■ ^ i>- . ■ ' - ■ 

THE. PEVELOPflCNT PROJECT ; 

The development occurred over, a four- 
year period and in two distinct phases- 
.The first, was the development of an 
in-'Corei nof'frillsi linear programming 
system- f/lve programmers completed " 
the development ^in . less 'than one year 

" The; code was refin^il over a two-year 
period by one programmer working on 

; a casual basis- ' . , ^. 

The second development phase involved ' 
t adding several 'major features to the 
'* system- These includecl an 'out-of-core ^ 
- capability^- mixecJ, integer and . parametric - 



programmi ng-i and a matrix-^ reduction, 
facility- The product v as requiredVo 
operate on three series 'of comput-ers, and 
run under "'tjfiree different operating 
systems- This developroent phas'e involved, 
as many, as 12 programmers over a 1-1/2 • 
year period- " The ob jecti^ve^of each phase 
was tti produce a commercially profitable, 
software product- 



THE DEVgLOPnENT TECHNIQUE S ' 



•Project staff organization was Similar to 
the. "chief programmer tearn" described by 
. .Baker and Hill.s -CI>: ' The ^ chief programmer 
N te*am consists of a chief programmer t a 
•backup pr^ogramffler, a project secretary-i 
and a pool of programmers* 

' The ;'chi-efT ^programmer is generally, the most 
senior' member of ,Jt.he development team- He 
is a working^ member of* the team with pver- 
^ all technical responsibility for the 
project- ''On this project^ his duties 
included system design-i assigning tasks ' 
to proj-ect membersi\and co-ordinating^ 
.'efforts of .inpli vidua! orogrammers. He 
^ 'also participated" in coding, and testing 
J ^ the system- :In the first phase of 

'development -I he produced, as- much code as 
any . other • project member- Zn the -second 
• . phase-i he .produced far less code- Ilore 
. time was spent on design and technical • 
administration of. the project.- * , 

^ ;The backup programmer assists the chief/. ^ 
programmer- and shares his dLtti'e^. He is 
Afamiliar w.^h the' entire' project and. is 
able; to assume chief programmer responsi- 
iiality at any time^ ^ ■ ' r V 

The pi^oject secretary maintain^^^^^m^ster 
library. of the^^system and doc)i4ne'ntation 
during development".^ The. secretary 
incorporates .new and corrective code in 
..ther System"^ keeps 3 recbrd^of all changes 
iSade-r and maintains listings of the "system. 
The^ project secretary described by Baker 
and Xlills "also was responsibile for 
^ mak-ing and keeping a record of all test " 
runs- In. this . development -i .project • . 
members too,k ttirns -handling the Tsecretar- 
^ial dtj^ies' .Only t.he most .significant 
test rtjns were .recorded ..during the 
development- - - ^ . 

System subroutines were written in f ORTRAN 
and assembly language-, Routines cri^tical 
to performance and ^hose handling packed - 
data were written in assembly -language- 
Remaining routines were written in FORTRAN-. 
Extensive use of FORTRAN^ eased the /prob- ' 
lems of . executing under different 
. operating ;sj'st«ms- 'Sufcjrq^utyinas could b'e 
. coded and- tested more' g^uickly using 
^^OR'^J^AN than using assembly language- ' 
. This left more time for writing perf orm- 
"ance critical: routines^ 1 

l>QCument'a4:iorl- Standards were specif ied, 
and enforced./ . At the -»st:art of the pro- 



■ ject-i each member was given a loose leaf • 
notebook fbr internal^ documentation- '" The 
• notebook was added to ^-s. subroutines' iJere 
• . defined and designed . and .updated as ; 

routines were modified- Internal documen 
' tation consisted of subroutine descrip- 
VtionsT communicatiorr region -CCRJ-jcell 
de^riptionsi-.. and array/file, formats- 

p. - 

Subroutine and CR" cell descr ipt ions were * 
^ maintained on the program library- This 
documentation was extracted from the. 
library. via computer program-i ^topied'and / 
. distributed to project members whenever 
changes, were m.ade- 

Subrou^tine documentation consisted df 
. comment cards with$i each routine- ile 

felt documentation was more likely to be 

.kept current when -irtcluded in thenroutine- 
. « Documentation was thea:up*dated at the same 

"time and. in- the same manner, as the \ 
-executable code- Two .levels of d-tfcumenta* 

tion.were required for subrouticfes- The- V 
> ^irst level \describes what the^putige* 
; does and how. it is .called- ' TFfes .lev^ 
. written, when' a .necessary oper^tCon ij 

defined- The ■ second level isTa step'.^^ 

stej5.^escription of how \t\e roiitine per^ 
^ form/ its function- This docuJentation 

is^ the result of subroutine, de'^ign , and 
. is equivalent to fl.ou.^charts- . Both levels 

-are written before the subroutine is codetj. 

CR cell definitions and documentation were 
•maintained as a data file on the system'' 
li-brai^y-^ . Subroutine" COnn ON statements 
were generated , from this, file by 'a com-^ 
puter prograiriL.-^ This program also produced 
printed documentation of the CR cells*'/ 

: *A^m^ima»l set of' coding standards were: "^^ ' 
established- The primary ;pupbse of these 

- Standards was to pr^ote simple.v^reacVable ' 
code- The standards were intentionally 

.;ninimal so they were ■ easily folloued- 
Sta'ndard F.ORTRAN calling sequences were ' ^ 
used for all suBroutines-' .Subroutine ; 
names begin with a (J or a J -if, a function 
subroutine- CR ;cells were named to 
.d«ifferentiate them from 'local • variables ' 
and to describe their/'^use. The f^rst 
character of a CR cell name s'pecifii&s 
whether it's: contents is alpha-i integer-i • 
or. real^ ' The second character of , the 
namfe sp&cifies its use- nThe^ name then " 
defi,nes the cell to be a/name-, switch,. 
ihdex-t count-i parameter-i or tolerance- 
Statement numbers in. FOR ?R AN routines - 

- are in ascending order - '-Each routine • 
was restricted to a single exit- 

'^tructured pr ogramming-»- top-down i^ro-^ 
g.rammingv and open^ ^programming were . ** 

considered for the- second development ' 
phase - ■ . ' / ' , . ' 

Structured .programming -Cl-.2,3>. generally - 
•does:-not allow.- 5 0 TO statements - "Rath&r-. ' 
all decislof^s are prbgrammed using IF- *. " 
THEN-En3E and. DO constKuc^ts- The -purpose^ ■ 
is to provided easily readable code by * 

. ^ * ■ ^ ■ — • .. 



avoiding complicated branches of control- 
The *lF-THEN-ELSE^onstructs serve to . 
define all conditions* under i^hich a 
section of code is, execut%?l- ^ ^ 

One routine was written using structured 
programming to evaluate itit -usefulness for 
. this development- The routine performed a -:>« 
complicated vector packing operation-^ -~ \ 

, . ■ ■ . - , , » L , 

Structured programming, rules were rigor-, 
' ously^ppXied for the investigation- The' 

• vtechniqiue did' not produce * the desired 

re^.ts/ Rather-iJ the structure had to be 

• forced upon the routine and detracted from 
its clarity and performance- The technique 
was not. used in remaining development- 
However-* the idea was, not entirely rejected-' 

- Reasonable efforts were made to avoid 

' unnecessary- backtracking and to allow t. 
. control to flow' from top to bottom- 
Sections of code which-could be entei*ed in 

• ''special ways !^tijere flagged with comment 
' '^caKds--' \ . ' 

" Top-down prQgramraing -Cl-r3> is a technique 
" to organize thfe development into levels^ • 
.. of detail* The basic inputs-i outputs^ anjd- 

• operation of ^ the system are first defined- 
Next a firs-t level of detail of operation 
is' a<Jded to the\ system. The process is ^ 

. i^TMeated until all levels, of detail are ' ^ 

- -defined- Eacfi" level can be designed^ ' * • 

codedi and- testcd independently of 
succeeding leva! s- The technique was not 
used in ^his development- We felt that , 
design of "an'ef f icient linear programming 
; * system, consists of several iterations 

a top-downi bcttom-t>p sequence- Perform- 
ance critical routirtfes are -dependent, on ' 
details of method and fil'e and array 
formats-*' These aspects ..need to be' formu- 
lated quite early in the .design process.- 
To a degree-i these details determine' the 

form of higher level routines- 

*■ * ' . « 

Open; programming- -Cl-i2> is a technique which 
was used in this development- With this 

*' technique-i each programmer's design-i 
docuracntationi' and' code^ is reviewed by ^ ' 
another project^ member i A thorough review 
Sv,houl'd result in improved readability of 

-documentation and code- The review"- should 
also uncover ef*f*ors that may be missed 

- dOring systenr testing- A thorough review 
also requires a considerable amount of - 
effort- In tKis^.development-i ail dapumen- 
tatiVn and approximately half the .code was 

^ reyiewed- Each, programmer -first harfl.design;'^ 
documentation >*e viewed by another • Pribram- ^ 
mer." After review and coding througb'. a^\^, 
clean compile-i code was similarly reviewed^ * 

• AdjditionariyT all design documentaticin' was 
Reviewed by eitl^r the chief' or backup 

pnogranfcer-: 

» . * * •■ ' 

Software verification is a difficurt and 
• costly part of ^ny development. It^^. . 
Nc^econbnically 'itifeasible-i" if not impossible . 

t' to f ind dil.',errors' in a software product- 
? , Nevertheless -I the softwar^e must work 

* cprrec^iyv^^arly all the time to "be useful 



and profitable- Nearly all software ■ 
. de.velopment techniques described:'above 
contribute to software reliability- . Three 
direct methods of verification were used 
in this development. 

The open programm^ii^g review was,; *hr . i st .. 
means of verification- Secondly ^'v(?'~-al, 
utility routines were : tested' inc ?^ " i^intly 
of the system in a simulated, envi'r^'ilSent- 

This was an^atterapt at, checking but all., 
paths through critical routines'- Thirdly^ 
the integra'ted system uas tested- ' Several 
models were~^ generated to test special 
conditions-: Xh6 bulk of Jihe testing-i 
however*-! relied on real world linear 
programming models for system checkout 



CONCLUSIONS 



.The project 'was completed oh schedule -i 
.costs ^uere within 105: of budget and per- 
formance and reliability- standards were 
met-' ' * ' \ ^ 

JJtocumenrtation and coding standards were 
considered /Critical to success of, the 
project- , iilithout these standards-T it 
uiuld not have been possible^to integrate 
* and debug the system on schedule- 
Additional lyi documentation produced 
during the first phase of develppment con- 
si^'derably eased the second phase ^effort.,- ... 
Sever al^ of the problems that? were encpun- . 
tered were in areas where standards were 
not closely followed- . . : ' 



Open programming appeared^c 

ifcmcovered bugs and 



.^^ o ^e Well 
worthwhile- , The reviQ4ii ^^"^ 
resulted in improvjedr^dccumentatibn-'" Ue« 

spGrTt,'t?^n''^tbi§ .review was ttfore 
ii^^tW t^st - phase ^ - 
['^indicated': reviewed^ode 
y. 1/3 the. errors of non- 
'-'The- quality of review 
oved ;if^. all done b^y the chief , 
rammer-: These two individ* _ 
familiar with* the . entire 
eVi '-this pi ace^Z additional'" 
on two critical resources* 



feel the tim 
than ma 
Initial 
had, apprj 
.^review! 
could 
or backu 
uals are m 
'system.' H 
^t-ime.^ deman 



Project statistics cohfirmed the theor.y * 
that /considerable effort 'is required to 
^^uce accurate estimates. The project 
^aStjwecT speeding ^a portion of the 
*^ri,ginal budget before producing final . 
: ^:iji.t estimates-". Table 1 compares .the 
^crcuracy 'of ^hese estimates to the* portion 
of work c^jnpljst^e. when 'estimates were made- 



V Table 1 
Errors-. in. Estimation of 

. • Portion. Complete 
Task. - Before Estimate 



r Tasks 

Error in 
Estimate 



ERIC: 



360 



A 
B 

C 
D 



By.' 



Xinder 2^'/. 
nii 
.un4er &nz 
over Ib/f 



The iarror In estiwatQ for task/A was cau;5ed 
by a change of; scope impose^d^tipon the pro- 
ject ry The nesligible error for task B 
shows that costs can be controlledi given 
a reasbn'able estimate* The error for Task 
«C resulted because insufficient time was 
.2^nt on the estimate* Task 1> was com-^ 
pleted with. less effort than estimated 
because a^ simpler, implementation was 
devised. : 

Fina^ly-T it should be mentioned that 
^^individual l^rogriammers*^ attitudes are key 
th^success of any development. Pro- 
are completed on time by programmers 
JO Jace determined to, finish on time* 
lity products are turned but by 
Jirammers^ho take pride in the quality 
of /their wor%* These .programmers cSn make 
efyfective use of new software development 
tcichniques- ' " .- ^ 
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. MATH "S^I^BiAMMlNG. COMPUTER CENTER- 

(A .personal •perspective* as seen from a foxhole). 

J: R. iSllison 
' . Mobil Oil Corp, 



The ''-fact "that there is a general, and on- 
going conflict between the users and ^- / 
'their computer center is generally, and 
i believia correctly, taken for granted. 
"^Xf^^you . Will accept this as ^ fact of 'life, 
letl^a-naiyze why this conflict exists. 

The^first, and I believe foremos^t, feasou^ 
is that each views the. other, aes .the 
proverbial "black box", aind, since there - 
is little personal contact with or real <- 
understanding of the other's problems or - 
motivations, conflicts arise. ' . , • 

One sourcp 'of conf licjt : is the: dif if erence - 
.in objectives. The primary objective -of 
a computer center' is often stated to be - 
to "maximize" throughput, .efficiency, etc.. 
This can be jJeasured as computer center, 
income (charges)"vs budget prediction, as 
how well 'j^b, schedules ar^met,. or as 
wall cldck hours' or CPU hours per day thel 
center ,rjans. ' Other popular me as tares are £ 
how many jots are run each ^ day and "per- 
cent CPU ■' utilization" .. . • .: 

The. r4ath\Programniing us'er.. usually has as 
a primary^ objective the provision of an 
answer to a question askedby some level 
of managements-pertaining ^to.. the economic 
coilsecruences of .^taking or^ not taking some 
cbxarse of action. .This quest\pn includ^ . ; ' 
.-defining the. -physical courses Sf^ action ■ 
which go with; the economics.. -In pro-- 
viding the above answer, or a-nswers, the * 
Math Programming user is seldom able to 
schedule his work since questions are^ 
addressied to I^im and "outside", data is _ 
prpvided to .him <bn a- time table which he 
cannot control and the answers are re- 
quired by. a predetermined date' (prefer- , 
"ably yest^tday) which is not under", his- 
control and ^ is of ten not under the. con- ■ 
trol of the person asking the question.'^ 
The - resulting -work load disrupts the com- 
palter center ^schedules with a sudden on- ■ 
slaught of computer, load as m^&toed by . ^\ 
^tPU time without a corresponding increase 
in the ninnber of jobs run. ' 

•The user's- habit of, preferably working., 
only days ■ {prime shift) compounds . the- ■ 
problem ! since he wants to wake and 



cor.reCt errors on tlie'.same day so h^ can 
submit 'more work^ that night or,, better 
yet, get more prime shift -runs that day. 
The Math . Progrcunming user is: also "prob- 
lem" oriented and looks on the computer 
center as the provider of a. tool recjuired 
to solve "his" problem so that he can 
provide an answer to a complex question. 
He views the center as a cost and as an 
obstacle to be overcome when it does not . 
heet his requirements* After "all, com- 
puter^c enters don't make money - only 
■ answers make, money . 



, Comparisons of the above statements show. ; 
areas of conflict betwee'h scheduled and 
xinscheduled work; speci£ic vs. broad objec- 
tives, and differencesr. in. objectives and 
measurement of effectiveness. 

The computer center Viewpoint can of ten be^ 
characterized as a responsibility for ^^^r-* 

-maintaining a complex operating syst©jn^*Snd- 
hafdwsure which must re^ond to the Memajids 
of a wide variety of users, none of whom • 

' understand the computer center's problems^- 
or how a computer actually- functions. 
They also appear to.belieye that Math 
Prograimming users ^re unreasonable people 
who can' shut down or stretch .put .job pro- 
cess!iiig for other.jusers. (especially^ 
schedViled users) -i^f they are'allowed the 
amount., of CPU time -they need. Unfortu- 
natel^-; these viewpoints often have a-^ 
large 'element of truth in them. S tr etch- 
ing «OAit of ^ job "processing\ causes . scheduled 
job^ miss their sched'uife^'^nd " ha S; been;.- . 
known to 'make accounting tyg>e users ... . 
extrepely unhappy with .the- 'o^mputer center. 
This ^ is important to the computer center 
because ■ accoxm ting .types are 'mucj:i more - 
nuifterous, are very adept and well trained 

. at writing ^complaining memorandums, and • 
the compizter ■centers are . often controlled " 
by accounting' types. If you^lJDn't like., 
the aiboye', just substitute technical types 
pr^whoever' has ibeeii causing yCu the most 
trouble and' it will; be just . as applicable^ 

The Math Programmer's. point of view is 
,that "we" haye.to obtain ""optimal" answers 
to questions posed by management and these 
answers feust^be available by a given. 



•.deadline. Unfortunately, we must expend, 
that liittited resource Jcnown as- time to ob- 
tain da*ta necessary to understand and cor- 
rectly • model the problem as stated ; we . 
must spend additional- time removing ' '•bugs'* 
from the model an*d revising the model to 

..;fit.the actual question ^s oppo-sed to what 
we'^ti^ought the question was and, f inally V • ' 

. we must. obtain sufficient timefron th6 ' 
computer center to allow us to ideet our 
answer deadline and thus stay out , of trou- 
ble.. This 4ast item is:;Jcnown as providing* 

.. reasonable" turnaround -and^ since it^'ls 
: at the end of the^time chaih^ is'-the 
source of .much, agony to all concerned. 

The>Mai:h ProgrSnmirig user', ^wheii asked by 
; ,the- center how much CMnputer time he will. 
; . . need, siiaply says . he 4oesn ' t ]«iow. If he 
knew how much time' he needed and how ^ 
runs, he would; know the answer and 
' not need any.. While there is^much 

in this, most computer centers -stil-. -.^.^c 
* predictable and well scheduled runs which 
. he^ has j.ust told them he dpesn't have. 

Now, with the above problems and conflicts 
. of, interest; how can we learn to co-exist 
with a computer center and get work accom- 
plished without gi^Hng in' to them. \1 be-- 
lieve that it is possible by defining your 

f eneral ^ require^entfjs in . terms of. the • 
Olio wing I . 

(1) CPU time required 'for prime shift. and 
' total shift. 

(2) Job turnaround; Analyze what you are" 
^ asking, for and be reasoriable. Turn- 
around isn't as simple as it sounds. 

space requirements. Perma- 
nent storage space is critical. Disk 
space is almost always in short 
.supply, expensive, and highly desir-.. 
able. Tape is plentiful, cheap, and 
^. less desirable since itimust be V 
mounted. System "scratch" space ^ 
-.(temporary space) is cils^o limited 
- ' ■ since it is usiially disk. . Remember 

that tapes can. be limited by the niam- 

■ ■ ber of tape drives available. .You 

must ^ be realistic with yoiar * space ; - 
demands * if you expect turnaround since 
multiprogxaroming^meahs that there are 

■ more users than just you .demanding 
space at any one time. Extreme ^re- 
qT|Lremen4:s ,C2Cs| delav your jobs. ^ ' ^ 

(4) MdEory requirAents' - memory, can 'Se ' 
. expensive and limiting. Be reasonable 

■ and careful, even if VS (Virtual 
Systems) is aroxmd. " * ^. 

(5) Math Programmingjcbdes - you should 
. , know more about which ^codes aie best 

-for yo*i than the computer center does, 
but be prepared .to- prove your point ■ of 
. view and to ju^ify the nrosts. . 

To define the^ above requirements in any- 
thing approaching a qu^intitative manner, ^ 
vou need, tools which -will allow >ou- to 
define where you are and where you wer e.' 
Reports oasea on the SystSns Management 
Function (SMF) or Xts equivalent are the ■ 



• -best ..that^T hS/e found. Don^t rely i^fi 
opinions, . feelings,, -informed- opiiiohs- 
etc ^They- areV|}nreliable, cannot/be\com- ' 
pared, and are ^Jah excellent booby . trap for 
the unwary, C^eful hand tabulation of 
data from your -runs is better, than nothing 
but veory time coffsuroing, especially if ybu 
collect enough data to be rejally useful. 

\ Now let-* S touch oh some of my 'personal 
biases and prejudices (I prefer: to call 
them measurements) which I swear by^ when 
■ -'- >>«?y. agree, w my. point of view.' and at 
when they disagreed 

(1) User turnaround - time measured from 
■ ■ ~- job in at the reader to finished job ' 
outpait available to the user. The 
user correctly loves this but be " 
careful. Printout priorities' and 
length of printout (lines and printer 
„ speed) can influence this measure and 
since this isn't truly controlled by 
the center, is not exactly a . fair - 
> measure^of computer center perfor- 
■ mance. 

(2) Center turnaround - time measured 
. . from job in at the reader to job 

finished by the initiator and turned 
over to the output queue, .1 consider 
this a measure of the center's ability 
to provide service. If includes time 
. in the input queue plus time in the 
computer and thus measures items 
-con trolled", by the center and is a 
^ measure of their ability to dc work. 

(3) Execution ratio - wall clock' time 
' from initiator start to initiator 

finished divided by problem program 
CPU time. This^ is an indication of 
the load on the CPU and can bem used 
as -aTi indicationi of whether or not 
you are gettung your ''fair- share of 
the computer.. It is- also useful in 
estimating h6w long your job will be 
in the computer. . - ■ 

The above three . measurements must be accu- 
mulated-, averaged over au reasonable time 
span, and- tHen plotted, so that you can 
see trencis or changes from whatever 
Xiormal" is or becomes.. 

Additional information which allows you 
to ..see. how your job load fits into the 
*S computer center .and its schedule are 

helpful. . Data on whaf percent of th'e - 
total CPU time/ charges, or jobs you 
^^^^^^'^^^ can be useful espiacially- if it - 
iSTsignificai^tly large. You shouldfiMso 
.be. aware of any special services the cen- 
' ter.does, or doesn't, perform for you.. 

Once you have all of thig information, 

Uyou should meet with computer center 
naanagement and try to -obtain commitments- 
- ^« agreements on levels of service which 
they will provide .to you. This written 

- .agreement will then provide a basis for 

- co-existence between you. . 



• •. . ' ■' OPERA TIONA L MANAGEMENT -OF' MATHEMATICAL 

^i.; . ■' . . G- Keuninerer ' 

"C^;^:: - / ^: -"^ v " union carbide 

-.'J ■ ' , CHEMICALS AND PLASTICS DIVISION , 

. . / Operations 'Planning Departroant 



, Chemicals and Plastics productions account fgr approximately 40 percent of Union 
Carbide's sales resulting in the movement of 19 billion pounds, at . production annually. 
This.ac-tivity involves more thaJi 6,000 products manufactured at approximately 15 plants 
and moved through more than 100 bulk terminals and warehouses?* We deal with more than 
SOfOOO industrial customers utilizing six deepwater vessels; JISO inland barges, and. more 
than ,8;/ 000 tank cars, hopper cars, and van boxes*' 



My purpose in describing U^i-on Carbide ' s production/distribution network is to 
highlight thfs need for. the use of mathematical optimization techniques in -effective 
planning.' Obviously, it is a' complex business, .with highly intergrated product/location 
'.characteristics in a changing business environment* 

^. The planning system used to help manage our chemical operation generally consists ^ 

of three phases: ' - . 

1. Gathering and processing data as _input -. to .the 'mathematical optimization 
*■ ■ system., . • ' ^ . ' " 

- \. The solution of a mathematical model to develop the optimal" production and 
\^ distribution plan for the period. v 

3, .Communication dif the results of the planning model -to the various functions > 
responsible for the operation.' ^ . / 

(See attached flowchart of plcoiriing cycle,:) ^ . " . 

■ - V '"^ ■ ■ ■ : 

The successful itianagement of a. pianning system depends on all. three of these- 
phases. Any one without, the othier /two is useless. . I will discuss each of the phases 
with" particular attention to the needs of the 'mathematical model. 

. -.■ " ^ ■ ■ ■' ■ / ^ ' ' ■ , ' 

The. first phase of ..the plamnin'g. system, the gathering of data, can be broken in 
two separate activities. The. first is standar^ data or that information which does not . 
.change very'' rapidly . Normally, a yearly review and update of this is * sufficient * The 
type of data incruded here is toriually budgeted cost, capacities, production factors^ etc 

, " > ' " ■ ' , f 

* The other class of input data consists of the more current operai:ing data which 
can change from day to day. Transportation cost and limitations, inyentpry strategies, 
production limitations and strategies, and ^"e sales forecast are examples of this 
type of input. : ^ ' 

This strategic inform^ition is the more difficult £md tile most critical of . the 
data need- It is gathered through direct interface with various operating groups. A 
good understanding of their" functions and objectives is necessary so that the input 
can be interpreted- Normally / people in operations are» not knowledgeable in the 
functioning of mathematically based planning systems. Because of this, all meetings to 
gather this input data must , be handled with the idea of getting the understanding, and 
-cooperation of these various other groups. It is my "^responsibility to interpret their - 
input and strategies and develop 'the proper model representation. This must be done . 
without putting unnecessary re;strictions in the model which will prevent • ttue . ; 
opt i TTU za t i-Qiy. . 

TPhe second phase of this process is' the -solution 6f a inathematical model , . .The 
particular model used by .Union Carbide's th^emical and Olefin Division consists of a 



J^^inea^ prpg^j^ conta-iliing. appa:6ximately 12 , OOO; variables and . 3V00;O ecJatiAsV * 

^arral^e^<^<^resent;a *r^w matei'ials/^int^rpediates,- ^d^^Ei^ 

S-^^^^^^^^^^'^P^-^^' "^?^s-'are linked to. .tbia/iinear 'inatrix.^ The' resulting mod^l^s 
.'• solvfiTd using a recurs xvfe technique on ah..T3M :'System7370 Model 168 / Mathematical Bro- " ' 
gramming System Exitended (MPSX) is used to. set up and solve the' linear portion. 
Depending on the number of iterations requved to-jreach optimum, . the computing tftme 
. required is normally^less than .two hours. It is import4nt to note that this ±s!^ 

batch type system. The pr, ob rem s involved with runnii:ig a b.atch system are. quitek'dif- 
• ferent than an-.c3n-line "System. ■ ■ ■ •. . ' - -^ v\'- 

. .^^ running this models we interia^ce.Vith two separa^^^^^ ■Computer ^ : 

Operations has , responsibility for.. hardw availability and priority. Systeins--Sui>p6rt" '• 
IS respons3.1>le for. supplying,prosr^ingV System maintenance/ or sbf tva re ^support if 
. needed^. ..Problems often develop jin-bbtainir^g.. necessary" priority-due taV the. siz of the 
.,paa»n4;^g system and the keerv^;competition for limited computer resources. Because of 
■ 5^^%'/^^' ^5.<^?sir.able to place^^^^^ runs on- production status. That is, should- 
,.v.vP€f- planned m advance and the support commitment of Computer;^ Operations^ obtSlhed. 

The demand on comp-uter resourced is cyclic. During^ par^^oular times oMthe month 
: or quarter, the demand- on the computer is particularly heavy. By planning your mior 
runs- during low' demand time or off^hours such as nights and weekends, it is possible to 
improve your turnaround time. ^ , 

• . -^yo^e involved, in programming should be aware of the importance of program 

efficiency. The' CPU time, required by a job affects what turnaround it will get. Jobs 
that st^ m the computer and slow down. the network are not run during high demand time ^ 
.Wherr running on virtuaL -storage , ^ as is the^case -in most large installations/' the ef- 
ficiency, of the program^ that is^ the ration of 'CPU time to elapsed. time, is extremely 
^ important- ,The longer a job stays in the computer^ the greater , the chance of system 
problems developing and- the premature cancellation of the .job. In addition, the 
operating. cost of running the system will be reduced. Various techniques can and 
should be used to improve program efficiency when designing a batch type" system • that 
will be operated regularly. . p . ' 

- Another 'important consideration is to insure that any important instruction to 

•uhe operator, such as mounting tapes or writing over tapes, should be gi-ven as close to 
the time the action is to be taken as. possible. • The manager: of the planning activity 
has little control over the terminal operators. In. addition > ■ the terminal operators 
are mvolved m many activities at the same time. Therefore,' it is necessary to mini- 
mize the opportunity for misunderstandings that may result in premature termination 
■ot .a run. In large systems, this can be extremely costly in terms^of cost and time. 

■At the present time, we are operating. our system at a remote terminal. This 
arrangement adds to • the difficulties mentioned- above . rt is impossible to be as - 
familiar with changes in software and hardware as .when operating directly from the' 
computer -center. In th^.past; we were located at' the computer center and a better 
understanding and ^.effective use of the computer was much easier- 



When developing and programming a system such as the one I have described, that 
IS,, one that will be run. routinely in support of a planning or operating function, it 
IS very important to involve the system analyst when. deciding such elements as the- 
number. Q^f variables, procedures for updating,, maintenance, .etc. It seems that time 
and tijme^ again we . are in a situation where we need to be able to analyze one more 
variableT'than the program will allow and the .limit' on. variables was purely an 
^f^^^^^^^ decision made, by .th^designer . When deciding the limit of program capa- 
bility, the operating analyst in a better position to determine the Tieed' for more 
or less flexibility. ■ - , . , . . ■ ^^^^^^ ' ^. . ' , * 

^ A very important part of the planning systent is the report writer;. While the 
system analyst may be able . to -interpret a L.P. matrix, the- need to review the- pre- 
liminary ^re suits with, other if unctions exist: This^ highlights 'the. need for reports 
that can be-used to review the ' solution and identify needed corrections. These reports 
must be readable by persons not familiar with computer output- Th^design of these 
preliminary reports .can have a great affect on the ease. -of. review^-and the reasonableness 
of the final plan'.. .5 ; ^ 

The third and "final stage of the total planning system is the communication of' 
.the results. As .1 seated earlier, all elements^ are equally important. The best de- 
signed program, with the best input is useless if .nothing' is done with 'the results 



« . ^ " .r-Ijami* generalising when,I say this but it is my. personal/ feelSn^/that efforts tdO ^ 
tise-jdireci: computer output/as the means of communicatirirg; the" results , to the variou^' ^ 
V;5^nctional, groups are.not( as effective as interpretation by an analyst,' We do use com- 

• <puter output when detail .is needed for specif;Lc cXctioh- But I do not feel it can 

; i'nterpx^t and*-e6inmunicate the results of such a complex planning system. A considerable 

• «fifort on the 'part, of system analysts and other experienced people ' is "needed . to-,. interpret 
the xesults/ and -expJ^in all of these significant .implications.. A knowledge- of the pro- • 

• ducts .and the distribution network is. as. important as computer experience. 

- . .t '/^Your*' reaction.- at thisi pbijat." may be; that jtiathematicaX pr<^gramming. plays 'no -part 
;. inr: .the* la st^. phase V . /ThiS;. is/ The understandin^/and cooperation; of 'the 

^- programmer, is esseni:xar t^*^h^ when evaluating : the l.P; solution. I' 

h^e -found'^tKatYtlie^ si^rj;L'f fcant : eli^^ >of the- solutj^on' are *ho^. straightforwcird- 

. "Usually ,f some . follow-j;up caseA^tudies or sensitiAri-ty analysj)^ is needed. ;•- The. mathematical^ 

programmer plays an importTant role .during these -studies. ' His advice and under s tevpding ? 

of the program logic -can save considejcable time .and result in a. more meaningful 




* i would like- to restate, my position on ' tbe final stage. ;:^ItM.s . my. opinion thi 
direct output : from the -.computer, is not the best mode of communicaffng^ the optimal 
solution. ""J-t is more important that the report writer capability V^upport the effortssJ ' - 
of a systems analyst in interpreting the results* ' v '"- >■ * 

Communication of. the plan can take -various forms depending on the audience, and . 
nature of the' activity /.itself;.... Normally, greater understanding "is obtained through 
graphs, tables, and diagrams/ Prose should be used to elaborate the results, A cleax 
statement of all assiimptions' and ^gnificant limits should be made. 

3l' One. of the most important points .1 would like to communicate is the type of result 
generally looked for. • I 'hope that an . understanding of this will help the mathematical - 
programmer to. do a better .job in using program logic. The most ■ iinportant Part of the 
solution is the direction/ a 'business activitjr-STioul^ take, and not - the eScact numbers . 
contained in the solution. The business environment cbanges so rapidly that it is nearly 
impossilDle to pin down exact values. A^goob understanding of tJie direction or strategy 
that should be appli^;d in critical areas sucfe as production/ inventory , distributioni^etc,^ 
is essential. Programming efforts should beN^ohe. with greater understanding of this fact. 

- , My objective during this presentation has been, to give you more insight into the : 
problems and techniques .of running a large mathematically programmed planning system. 

.-hope .that this knowledge; will help^you or .give you a better' feel for some of the .^critical 
elements £n programs 'supporting large operating^ systems. I'm sure that a slightly dif-- . 

-ferent viewpoint would be presented if the system under discussion was an on-line . 
system. I do not have much experience with on— line systems and you should keep in mind " 
that all of my discussion is^based on batch type systems. ■ ■ . 

- ■■ ■ \ . \ ■ ' ■ ^ - : ; V . ^ , - . - . 

I support your efforts in trying to* create better understanding between system 
designers and System operators. .I.'m sure that all of ; us will be able to do^a better job 
if we apply ^me of the experience gained through^ understanding the other person's 
problems. 



eric; - • : • 



The Biscuit Division' of the Nabisco Corpor,ation presently '^6 3:>akeries/ each 

capable of' producing some ; but^nbt all/ of approximately 750' diff^^^tit products. . Ship- 
ping', branches jJerform the wareftouse* and distribution functions . re^^^^ed after the' bakery 
fhas produced and" packaged 'the products. They supply products to sales branches, 

of which there are approximately 225 located throughout the o^Lte^ ^tates.^ . 

Estimates of sales by product., 'QM^^ a. 12 week- horizon, a-re m^^^ at each sale's, 
branch. • These are sent by telecoinroitfnication equipment to a c^ntir^^ ^-ocation where they 
are converted Into prciduction Requirements which are then aliocat^ t:o individual pi'O- 
duction facilities; i.e., ovens;^ incing machines and packing equiP'^^iit- The resulting 
production plaii specifies the cimount of each product to proc^u^e ^^ch^ facility for ^ach 
sales ^branch during each of the four-week periods in the horizon, ^?Ke production plan 
minimizes' thie total variable . cost of prpduction arid distribution ^^^ie". insuring its 
production feasibility.' 

This paper concentrates on the operational aspects of th^ ^Y^?|^' use of the 

computer amd its^ relationship to the real world,, both- from th^ fx^^ unit and the % 
division management ''po^nt of view. . ' ^ ' . * * 
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